[wikireader]Suggestions for next steps on software

[David Samblas Martinez]

Though on how the evolution of the soft of the wikireader must be, I
have oredered the sugesstions by impact on functionality/easy to
implement ratio, of course under a totally  non-hardcore developer and
only-one-week user criteria.
*First and prioritary,
allow have multiple languages on same sdcard, I think a one easy
approach is to have a selection menu on boot with the available
languages on the sdcard (looking at the sufix of the pedia files
"pedia_es", "pedia_de", "pedia_pt"..... ) present a bare text menu
with the items, select,  and then operate as usual.
More complex things like changing of language inside a topic or
include non wikipedia content to  that menu using a config file can be
implemented later on.

*Second
adaptation of the keyboard for internationalization, I suggest ,as
quick fix, the introduction of a third keyboard screen with the
special chars like  ñ, ç, ... add here the common chars on  other
european languages, to simplify not include accentuated vocals and
consonants , when searching a word you will select the correct
accentuated one on the list. Cirilic, chinese, japanese, arabic and
other completly diferent alphabets is a full redo of the keyboard,
there are important to have in  but I don't describe his
implementation and be selectable as quickfix...

*Third
Use long press of the buttons search, and history to:
long press on search = launch other apps, present a menu (loaded from
a config file to launch apps like  the calculator(calc.elf file
included in the sdcard)and the multiple apps people will start to
develop on this platform, (note taking app will be helpful for
example)

I think change the behavior of the history short press as back is more
useful than actual one, and on long Press present a menu to
back/forward/view history/change content menu

Random is funny as it is please dont touch :)

*Forth:
Include images , I know  this will increase a lot both the space on
sdcard and cpu load of the device, but they can be implemented as
link(pic) in the main content and once clicked open them at full
screen , and pressing the history(back) button and return to content
, same aproach with formulas. zoom in/out maybe aviable from a long
press on the screen with a pop up menu, a fixed scales and max image
size has to be defined on what the device can handle. of course this
image strored in the sd are b/w bitmaps, the host is the responsable
of make the render of the original ones to this format. Maybe formulas
can be rendered on the device if they are stored in latex on the
wikipedia and the Wikireade can handle this or has to be redered as
images on the host it it can't be done.

What do you think about?



David Reyes Samblas Martinez
http://www.tuxbrain.com
Open ultraportable & embedded solutions
Openmoko, Openpandora,  Arduino
Hey, watch out!!! There's a linux in your pocket!!!

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Doug Jones]

David Reyes Samblas Martinez wrote:

> Though on how the evolution of the soft of the wikireader must be, I
> have oredered the sugesstions by impact on functionality/easy to
> implement ratio, of course under a totally  non-hardcore developer and
> only-one-week user criteria.
> *First and prioritary,
> allow have multiple languages on same sdcard, I think a one easy
> approach is to have a selection menu on boot with the available
> languages on the sdcard (looking at the sufix of the pedia files
> "pedia_es", "pedia_de", "pedia_pt"..... ) present a bare text menu
> with the items, select,  and then operate as usual.
> More complex things like changing of language inside a topic or
> include non wikipedia content to  that menu using a config file can be
> implemented later on.

I notice that the current implementation fits within the old 8.3 DOS
file naming scheme (except that both upper case and lower case is used).
  Is this done to avoid the FAT patent issue?

If so, then we need to devise a naming scheme that fits within that...
perhaps something like

wwwwllnn.ext


where:

wwww indicates which wiki

ll   indicates which language

nn   indicates which alphabetical section  (I assume the articles are
grouped according to first letter, 00 through 26 for A-Z and other
characters...   or is this wrong?)


Other alphabets have different numbers of letters, so do we sometimes
need more than two digits?

And some languages don't use alphabets...   has somebody already worked
out a general scheme for breaking up the files, and is this documented
somewhere?


We have to break up the data into chunks somehow.  FAT has a 4GB file
limit, as I recall...

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Pike-2]

Hi

.. I'm not sure if it has been mentioned - didnt follow
all the buzz on the list ..

.. and it's not a software update ..

.. but I initially expected Wikireader to have GPS
onboard and simply tell you about your surroundings.
There are 1000s of GPS annotated entries on
Wikipedia ? I see all those 'W' links in Google maps.
A perfect travelguide. WikiTravelMate.


$2c,
*-pike

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Doug Jones]

Doug Jones wrote:

> David Reyes Samblas Martinez wrote:
>> Though on how the evolution of the soft of the wikireader must be, I
>> have oredered the sugesstions by impact on functionality/easy to
>> implement ratio, of course under a totally  non-hardcore developer and
>> only-one-week user criteria.
>> *First and prioritary,
>> allow have multiple languages on same sdcard, I think a one easy
>> approach is to have a selection menu on boot with the available
>> languages on the sdcard (looking at the sufix of the pedia files
>> "pedia_es", "pedia_de", "pedia_pt"..... ) present a bare text menu
>> with the items, select,  and then operate as usual.
>> More complex things like changing of language inside a topic or
>> include non wikipedia content to  that menu using a config file can be
>> implemented later on.
>
>
> I notice that the current implementation fits within the old 8.3 DOS
> file naming scheme (except that both upper case and lower case is used).
>   Is this done to avoid the FAT patent issue?
>
> If so, then we need to devise a naming scheme that fits within that...
> perhaps something like
>
> wwwwllnn.ext
>
>
> where:
>
> wwww indicates which wiki
>
> ll   indicates which language
>
> nn   indicates which alphabetical section  (I assume the articles are
> grouped according to first letter, 00 through 26 for A-Z and other
> characters...   or is this wrong?)
>
>
> Other alphabets have different numbers of letters, so do we sometimes
> need more than two digits?
>
> And some languages don't use alphabets...   has somebody already worked
> out a general scheme for breaking up the files, and is this documented
> somewhere?
>
>
> We have to break up the data into chunks somehow.  FAT has a 4GB file
> limit, as I recall...

Actually it seems a bit more complicated than this.

The language designations used within Wikipedia are not just limited to
two characters  --  these aren't just country codes.

Most languages there are designated by a two-letter code, but some are
three letters, and go all the way up to "simple" for simple English and
"cbk-zam" for Chavacano de Zamboanga.

Clearly we should be aiming for a system that can accommodate all
available versions of Wikipedia, as well as those yet to be implemented.


The current implementation uses a flat directory structure, with no
subdirectories.  Is there some compelling reason for this?

We could put each separate wiki into its own directory.  This would make
it much easier to fit everything within the 8.3 naming scheme (assuming
that this scheme really is a requirement).  It would also make it easier
for the user to copy a new wiki onto the card;  they just have to copy
one folder, instead of keeping track of dozens of files.

This would also eliminate the need for a config file at the root that
would need to be updated as wikis are added or removed.  Instead, the
boot code scans the root, finds all the directories, looks inside each
one for a short metadata file that contains the description for that
wiki (in the appropriate language of course), and then uses that data to
build the first menu that the user sees.

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[David Samblas Martinez]

2009/10/30 Doug Jones <[hidden email]>:

> Doug Jones wrote:
>> David Reyes Samblas Martinez wrote:
>>> Though on how the evolution of the soft of the wikireader must be, I
>>> have oredered the sugesstions by impact on functionality/easy to
>>> implement ratio, of course under a totally  non-hardcore developer and
>>> only-one-week user criteria.
>>> *First and prioritary,
>>> allow have multiple languages on same sdcard, I think a one easy
>>> approach is to have a selection menu on boot with the available
>>> languages on the sdcard (looking at the sufix of the pedia files
>>> "pedia_es", "pedia_de", "pedia_pt"..... ) present a bare text menu
>>> with the items, select,  and then operate as usual.
>>> More complex things like changing of language inside a topic or
>>> include non wikipedia content to  that menu using a config file can be
>>> implemented later on.
>>
>>
>> I notice that the current implementation fits within the old 8.3 DOS
>> file naming scheme (except that both upper case and lower case is used).
>>   Is this done to avoid the FAT patent issue?
>>
>> If so, then we need to devise a naming scheme that fits within that...
>> perhaps something like
>>
>> wwwwllnn.ext
>>
>>
>> where:
>>
>> wwww indicates which wiki
>>
>> ll   indicates which language
>>
>> nn   indicates which alphabetical section  (I assume the articles are
>> grouped according to first letter, 00 through 26 for A-Z and other
>> characters...   or is this wrong?)
>>
>>
>> Other alphabets have different numbers of letters, so do we sometimes
>> need more than two digits?
>>
>> And some languages don't use alphabets...   has somebody already worked
>> out a general scheme for breaking up the files, and is this documented
>> somewhere?
>>
>>
>> We have to break up the data into chunks somehow.  FAT has a 4GB file
>> limit, as I recall...
>
>
> Actually it seems a bit more complicated than this.
>
> The language designations used within Wikipedia are not just limited to
> two characters  --  these aren't just country codes.
>
> Most languages there are designated by a two-letter code, but some are
> three letters, and go all the way up to "simple" for simple English and
> "cbk-zam" for Chavacano de Zamboanga.
>
> Clearly we should be aiming for a system that can accommodate all
> available versions of Wikipedia, as well as those yet to be implemented.
>
>
> The current implementation uses a flat directory structure, with no
> subdirectories.  Is there some compelling reason for this?
>
> We could put each separate wiki into its own directory.  This would make
> it much easier to fit everything within the 8.3 naming scheme (assuming
> that this scheme really is a requirement).  It would also make it easier
> for the user to copy a new wiki onto the card;  they just have to copy
> one folder, instead of keeping track of dozens of files.
>
> This would also eliminate the need for a config file at the root that
> would need to be updated as wikis are added or removed.  Instead, the
> boot code scans the root, finds all the directories, looks inside each
> one for a short metadata file that contains the description for that
> wiki (in the appropriate language of course), and then uses that data to
> build the first menu that the user sees.
>
> _______________________________________________
> Openmoko community mailing list
> [hidden email]
> http://lists.openmoko.org/mailman/listinfo/community
>

I like this approach, just remind that as far I see the code and by
comments of the devs, the kernel implements the bare just enough to
read files, so I think directories are not implemented at all that's
why all is on root directory so at least basic hierarchical filesystem
has to be implemented before we can do this solution. But directories
will easy the organization of pictures too

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Think-Free]

.. but I initially expected Wikireader to have GPS
onboard and simply tell you about your surroundings.
There are 1000s of GPS annotated entries on
Wikipedia ? I see all those 'W' links in Google maps.
A perfect travelguide. WikiTravelMate.

Hi !

I'm starting a project like this next week for the freerunner ;) 

Stay tuned ;)

 

$2c,
*-pike

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

--
------------------------------

Openmoko phone gui :

http://www.qalee.org


_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Doug Jones]

[snip]
> I like this approach, just remind that as far I see the code and by
> comments of the devs, the kernel implements the bare just enough to
> read files, so I think directories are not implemented at all that's
> why all is on root directory so at least basic hierarchical filesystem
> has to be implemented before we can do this solution. But directories
> will easy the organization of pictures too


Good point.

So two important questions to be answered, before we get any further
into this:

(1) Has OpenMoko made the policy decision that filenames will be limited
to 8.3?


(2) How complicated will it be to implement subdirectory support?



Note that only one subdirectory level is really needed to implement what
has already been suggested.

The current implementation contains 81 files, totaling 4.2GB for the
English version.  Nearly all of that is in the big wiki data files
(pedia*).  The other files, the ones you get when you make install,
comprise 49 files and only 18MB, and most of that is fonts (which are
often different for different languages).

We could adopt a brain-swap approach:  After bootup, the user selects
one wiki and then the app switches to the selected subdirectory and
considers that to be the root until the next cold boot.  All 81 files
for that particular wiki and language would be in that subdirectory,
including the big wiki data files and the fonts and the remaining files
(45 files, only 381KB, and this includes ALL of the executables!)  While
the single app is running, it would not have to access (or even know
about) anything outside its current directory, so no filesystem calls
relating to directory navigation would be needed within that particular
kernel.elf.  Only the initial wiki selection app (we would have to write
one) would have to understand subdirectories, and only to one level deep.


_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[David Samblas Martinez]

2009/10/30 Doug Jones <[hidden email]>:

> [snip]
>> I like this approach, just remind that as far I see the code and by
>> comments of the devs, the kernel implements the bare just enough to
>> read files, so I think directories are not implemented at all that's
>> why all is on root directory so at least basic hierarchical filesystem
>> has to be implemented before we can do this solution. But directories
>> will easy the organization of pictures too
>
>
> Good point.
>
> So two important questions to be answered, before we get any further
> into this:
>
> (1) Has OpenMoko made the policy decision that filenames will be limited
> to 8.3?
>
>
> (2) How complicated will it be to implement subdirectory support?
>
>
>
> Note that only one subdirectory level is really needed to implement what
> has already been suggested.
>
> The current implementation contains 81 files, totaling 4.2GB for the
> English version.  Nearly all of that is in the big wiki data files
> (pedia*).  The other files, the ones you get when you make install,
> comprise 49 files and only 18MB, and most of that is fonts (which are
> often different for different languages).
>
> We could adopt a brain-swap approach:  After bootup, the user selects
> one wiki and then the app switches to the selected subdirectory and
> considers that to be the root until the next cold boot.  All 81 files
> for that particular wiki and language would be in that subdirectory,
> including the big wiki data files and the fonts and the remaining files
> (45 files, only 381KB, and this includes ALL of the executables!)  While
> the single app is running, it would not have to access (or even know
> about) anything outside its current directory, so no filesystem calls
> relating to directory navigation would be needed within that particular
> kernel.elf.  Only the initial wiki selection app (we would have to write
> one) would have to understand subdirectories, and only to one level deep.

mmm this aproach will not complicate too much if after more apps than
a reader are implemented? It would not be better to have a even a one
level dir fs? nevertheless you will implement it for the starting
menu, why not do it directly in the kernel and allow other apps take
advantage of this implementation?
Again talking from the most totally inexperience.
>
>
> _______________________________________________
> Openmoko community mailing list
> [hidden email]
> http://lists.openmoko.org/mailman/listinfo/community
>

David Reyes Samblas Martinez
http://www.tuxbrain.com
Open ultraportable & embedded solutions
Openmoko, Openpandora,  Arduino
Hey, watch out!!! There's a linux in your pocket!!!

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Paul Fertser]

Hi,

David Reyes Samblas Martinez <[hidden email]> writes:
> *First and prioritary,
> allow have multiple languages on same sdcard

TBH i can't really understand the desire to have encyclopedic articles
written in other languages. It's encyclopedia, not literature after
all! Why not concentrate on correctness and volume instead of
translating from one language to another equivalent (for this
purposes)? It's just happened so that English became an
internationally understood way of communication, what is the benefit
in trying so hard to do something to alter that?

I'd prefer having a good English dictionary integrated instead.

--
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software!
mailto:[hidden email]

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[arne anka]

> purposes)? It's just happened so that English became an
> internationally understood way of communication, what is the benefit
> in trying so hard to do something to alter that?

- english wikipedia does not cover all topics
- different languages provide different quality of articles
- because there _are_ different versions of wikipedia already

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Doug Jones]

David Reyes Samblas Martinez wrote:

> 2009/10/30 Doug Jones <[hidden email]>:
>> [snip]
>>> I like this approach, just remind that as far I see the code and by
>>> comments of the devs, the kernel implements the bare just enough to
>>> read files, so I think directories are not implemented at all that's
>>> why all is on root directory so at least basic hierarchical filesystem
>>> has to be implemented before we can do this solution. But directories
>>> will easy the organization of pictures too
>>
>> Good point.
>>
>> So two important questions to be answered, before we get any further
>> into this:
>>
>> (1) Has OpenMoko made the policy decision that filenames will be limited
>> to 8.3?
>>
>>
>> (2) How complicated will it be to implement subdirectory support?
>>
>>
>>
>> Note that only one subdirectory level is really needed to implement what
>> has already been suggested.
>>
>> The current implementation contains 81 files, totaling 4.2GB for the
>> English version.  Nearly all of that is in the big wiki data files
>> (pedia*).  The other files, the ones you get when you make install,
>> comprise 49 files and only 18MB, and most of that is fonts (which are
>> often different for different languages).
>>
>> We could adopt a brain-swap approach:  After bootup, the user selects
>> one wiki and then the app switches to the selected subdirectory and
>> considers that to be the root until the next cold boot.  All 81 files
>> for that particular wiki and language would be in that subdirectory,
>> including the big wiki data files and the fonts and the remaining files
>> (45 files, only 381KB, and this includes ALL of the executables!)  While
>> the single app is running, it would not have to access (or even know
>> about) anything outside its current directory, so no filesystem calls
>> relating to directory navigation would be needed within that particular
>> kernel.elf.  Only the initial wiki selection app (we would have to write
>> one) would have to understand subdirectories, and only to one level deep.
>
> mmm this aproach will not complicate too much if after more apps than
> a reader are implemented? It would not be better to have a even a one
> level dir fs? nevertheless you will implement it for the starting
> menu, why not do it directly in the kernel and allow other apps take
> advantage of this implementation?
> Again talking from the most totally inexperience.

These are good questions.  But first we need to answer question #1
above.  If we do indeed limit ourselves to 8.3 file names, that also can
complicate a lot that we might want to do later, especially if we are
limited to a flat directory.


When we are thinking about future apps to put on this device, we ought
to be careful to keep in mind its current limitations, and what it would
take to go beyond them.  It has no actual operating system, as most
people understand this term.  Each app has to be almost completely
self-contained.  There are no services to allow apps to talk to each
other, and indeed only one app can run at a time.  When the WikiReader
app is running, it is in complete control of the device and nothing else
can run until you reboot the device.  (The WikiReader app is called
kernel.elf for a reason.)

Changing this would require putting something resembling a real
operating system on there.  This is certainly possible, but I suspect it
is way, way beyond the scope of anything we have been talking about so far!


_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[David Samblas Martinez]

2009/10/30 Paul Fertser <[hidden email]>:
> Hi,
>
> David Reyes Samblas Martinez <[hidden email]> writes:
>> *First and prioritary,
>> allow have multiple languages on same sdcard
>
> TBH i can't really understand the desire to have encyclopedic articles
> written in other languages.
let me gess... your native  language is english isn't it? :P
>It's encyclopedia, not literature after
> all! Why not concentrate on correctness and volume instead of
> translating from one language to another equivalent (for this
> purposes)? It's just happened so that English became an
> internationally understood way of communication, what is the benefit
> in trying so hard to do something to alter that?
as arne has pointed is not just translating each wikipedia in
different language is a different wikipedia and local topics as
geography and cultural topics are more rich in the local language than
in a foreign one.
Also you must face a fact... not everyone in a non english country
speaks/reads/or even have listen ever  english! yes! it's true believe
me , or travel a little more (outside the tourist circuits of course)
:)

Also comercial reasons the more lenguages can be integrated the more
maket you will arrive.
>
> I'd prefer having a good English dictionary integrated instead.
I prefer a good Spanish one :) I gess if RAE (Royal Academy of
Spanish) has an xml to parse :P
>

David Reyes Samblas Martinez
http://www.tuxbrain.com
Open ultraportable & embedded solutions
Openmoko, Openpandora,  Arduino
Hey, watch out!!! There's a linux in your pocket!!!
> --
> Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software!
> mailto:[hidden email]
>

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[David Samblas Martinez]

2009/10/30 Doug Jones <[hidden email]>:

> David Reyes Samblas Martinez wrote:
>> 2009/10/30 Doug Jones <[hidden email]>:
>>> [snip]
>>>> I like this approach, just remind that as far I see the code and by
>>>> comments of the devs, the kernel implements the bare just enough to
>>>> read files, so I think directories are not implemented at all that's
>>>> why all is on root directory so at least basic hierarchical filesystem
>>>> has to be implemented before we can do this solution. But directories
>>>> will easy the organization of pictures too
>>>
>>> Good point.
>>>
>>> So two important questions to be answered, before we get any further
>>> into this:
>>>
>>> (1) Has OpenMoko made the policy decision that filenames will be limited
>>> to 8.3?
>>>
>>>
>>> (2) How complicated will it be to implement subdirectory support?
>>>
>>>
>>>
>>> Note that only one subdirectory level is really needed to implement what
>>> has already been suggested.
>>>
>>> The current implementation contains 81 files, totaling 4.2GB for the
>>> English version.  Nearly all of that is in the big wiki data files
>>> (pedia*).  The other files, the ones you get when you make install,
>>> comprise 49 files and only 18MB, and most of that is fonts (which are
>>> often different for different languages).
>>>
>>> We could adopt a brain-swap approach:  After bootup, the user selects
>>> one wiki and then the app switches to the selected subdirectory and
>>> considers that to be the root until the next cold boot.  All 81 files
>>> for that particular wiki and language would be in that subdirectory,
>>> including the big wiki data files and the fonts and the remaining files
>>> (45 files, only 381KB, and this includes ALL of the executables!)  While
>>> the single app is running, it would not have to access (or even know
>>> about) anything outside its current directory, so no filesystem calls
>>> relating to directory navigation would be needed within that particular
>>> kernel.elf.  Only the initial wiki selection app (we would have to write
>>> one) would have to understand subdirectories, and only to one level deep.
>>
>> mmm this aproach will not complicate too much if after more apps than
>> a reader are implemented? It would not be better to have a even a one
>> level dir fs? nevertheless you will implement it for the starting
>> menu, why not do it directly in the kernel and allow other apps take
>> advantage of this implementation?
>> Again talking from the most totally inexperience.
>
>
> These are good questions.  But first we need to answer question #1
> above.  If we do indeed limit ourselves to 8.3 file names, that also can
> complicate a lot that we might want to do later, especially if we are
> limited to a flat directory.
>
>
> When we are thinking about future apps to put on this device, we ought
> to be careful to keep in mind its current limitations, and what it would
> take to go beyond them.  It has no actual operating system, as most
> people understand this term.  Each app has to be almost completely
> self-contained.  There are no services to allow apps to talk to each
> other, and indeed only one app can run at a time.  When the WikiReader
> app is running, it is in complete control of the device and nothing else
> can run until you reboot the device.  (The WikiReader app is called
> kernel.elf for a reason.)
>
> Changing this would require putting something resembling a real
> operating system on there.  This is certainly possible, but I suspect it
> is way, way beyond the scope of anything we have been talking about so far!

Yes, I had missed the point of the monolitic app, so then an easy
approach to make different apps (menus,calculator, note taking, basic
drawing, ...) is to enrich  it in the reader app itself this way maybe
some services (read services as already implemented functions in the
code of the reader app) can be used by the little apps, understanding
than this little apps are not independent apps but just another screen
of the same big app.

>
>
> _______________________________________________
> Openmoko community mailing list
> [hidden email]
> http://lists.openmoko.org/mailman/listinfo/community
>
David Reyes Samblas Martinez
http://www.tuxbrain.com
Open ultraportable & embedded solutions
Openmoko, Openpandora,  Arduino
Hey, watch out!!! There's a linux in your pocket!!!

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Paul Fertser]

David Reyes Samblas Martinez <[hidden email]> writes:
> 2009/10/30 Paul Fertser <[hidden email]>:
>> David Reyes Samblas Martinez <[hidden email]> writes:
>>> *First and prioritary,
>>> allow have multiple languages on same sdcard
>>
>> TBH i can't really understand the desire to have encyclopedic articles
>> written in other languages.
>
> let me gess... your native  language is english isn't it? :P

Wrong :P I learnt English a little bit at school and later at the
University. I guess it's obvious for a native speaker judging by my
far from ideal writing skills.

>> It's encyclopedia, not literature after
>> all! Why not concentrate on correctness and volume instead of
>> translating from one language to another equivalent (for this
>> purposes)? It's just happened so that English became an
>> internationally understood way of communication, what is the benefit
>> in trying so hard to do something to alter that?
> as arne has pointed is not just translating each wikipedia in
> different language is a different wikipedia and local topics as
> geography and cultural topics are more rich in the local language than
> in a foreign one.

It's true, i agree. But i tend to look for non-local topics on >90% of
occassions. Also in my experience looking for local topics is usually
more rewarding when i do simple googling instead of reading localized
wikipedia.

> Also you must face a fact... not everyone in a non english country
> speaks/reads/or even have listen ever  english! yes! it's true believe
> me , or travel a little more (outside the tourist circuits of course)
> :)

I think volunteers that write wikipedia articles do that for the
benefit of the people who want to learn. And those who do can learn
enough English to understand articles, it'd also have other numerous
benefits obviously.

> Also comercial reasons the more lenguages can be integrated the more
> maket you will arrive.

Heh, commercial reasons are something i tend to overlook usually :)

--
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software!
mailto:[hidden email]

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[David Samblas Martinez]

2009/10/30 Paul Fertser <[hidden email]>:

> David Reyes Samblas Martinez <[hidden email]> writes:
>> 2009/10/30 Paul Fertser <[hidden email]>:
>>> David Reyes Samblas Martinez <[hidden email]> writes:
>>>> *First and prioritary,
>>>> allow have multiple languages on same sdcard
>>>
>>> TBH i can't really understand the desire to have encyclopedic articles
>>> written in other languages.
>>
>> let me gess... your native  language is english isn't it? :P
>
> Wrong :P I learnt English a little bit at school and later at the
> University. I guess it's obvious for a native speaker judging by my
> far from ideal writing skills.

Obvously I'm not a native speaker either due my lack of criteria judging :P

>
>>> It's encyclopedia, not literature after
>>> all! Why not concentrate on correctness and volume instead of
>>> translating from one language to another equivalent (for this
>>> purposes)? It's just happened so that English became an
>>> internationally understood way of communication, what is the benefit
>>> in trying so hard to do something to alter that?
>> as arne has pointed is not just translating each wikipedia in
>> different language is a different wikipedia and local topics as
>> geography and cultural topics are more rich in the local language than
>> in a foreign one.
>
> It's true, i agree. But i tend to look for non-local topics on >90% of
> occassions. Also in my experience looking for local topics is usually
> more rewarding when i do simple googling instead of reading localized
> wikipedia.

yes bun in this in case of wikireader you have no Saint Google to pray
you are offline so the more sources you can store in the card the
better and the first easy step is to include more wikipedia.

>
>> Also you must face a fact... not everyone in a non english country
>> speaks/reads/or even have listen ever  english! yes! it's true believe
>> me , or travel a little more (outside the tourist circuits of course)
>> :)
>
> I think volunteers that write wikipedia articles do that for the
> benefit of the people who want to learn. And those who do can learn
> enough English to understand articles, it'd also have other numerous
> benefits obviously.

I don't know where you are but in Spain we are little far to
generalize this as a fact .English level is far from desiderable in a
european country, things are changing slowly, now english is learned
or at least listened in very early stages in school , my three year
daughter two teachers one of thems is an only english speaking teacher
and she is starting to understand some easy orders and sentences I
hope when this generation grow things will be different here.
>
>> Also comercial reasons the more lenguages can be integrated the more
>> maket you will arrive.
>
> Heh, commercial reasons are something i tend to overlook usually :)
more units sold more users, more resources to OM and his friends to to
still investing in more cool open&free ideas :) as companies we can't
overlook this ;)
>
> --
> Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software!
> mailto:[hidden email]
>


David Reyes Samblas Martinez
http://www.tuxbrain.com
Open ultraportable & embedded solutions
Openmoko, Openpandora,  Arduino
Hey, watch out!!! There's a linux in your pocket!!!

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Doug Jones]

[snip]
>
> Yes, I had missed the point of the monolitic app, so then an easy
> approach to make different apps (menus,calculator, note taking, basic
> drawing, ...) is to enrich  it in the reader app itself this way maybe
> some services (read services as already implemented functions in the
> code of the reader app) can be used by the little apps, understanding
> than this little apps are not independent apps but just another screen
> of the same big app.
>


Actually, the possibility that interests me the most is the idea of
making it easy to put additional content into the reader.  The current
software is good at displaying textual content, and good at searching
for text as well.  The hardware is less than optimal for the other types
of apps you mentioned  --  but it's a good reader.

There are many sources of text that one might want to carry around in
such a device, not just Wikipedia.  E-books, websites, mail archives,
other wikis...   all we have to do is build the tools to convert the
content into a format the device can understand.  This is much easier
than writing new code to run on the device.


For those people who want to write code that runs on the device, there
are plenty of things that would make it an even better reader.  But I
expect to concentrate on the WikiReader tools that run on other computers.




_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Sean Moss-Pultz]

On Sat, Oct 31, 2009 at 3:46 AM, Doug Jones <[hidden email]> wrote:

> [snip]
>> I like this approach, just remind that as far I see the code and by
>> comments of the devs, the kernel implements the bare just enough to
>> read files, so I think directories are not implemented at all that's
>> why all is on root directory so at least basic hierarchical filesystem
>> has to be implemented before we can do this solution. But directories
>> will easy the organization of pictures too
>
>
> Good point.
>
> So two important questions to be answered, before we get any further
> into this:
>
> (1) Has OpenMoko made the policy decision that filenames will be limited
> to 8.3?

We're using FAT.

> (2) How complicated will it be to implement subdirectory support?

Not a problem. We're going to do something like this when we support
more languages.

> Note that only one subdirectory level is really needed to implement what
> has already been suggested.
>
> The current implementation contains 81 files, totaling 4.2GB for the
> English version.  Nearly all of that is in the big wiki data files
> (pedia*).  The other files, the ones you get when you make install,
> comprise 49 files and only 18MB, and most of that is fonts (which are
> often different for different languages).

Correct.

> We could adopt a brain-swap approach:  After bootup, the user selects
> one wiki and then the app switches to the selected subdirectory and
> considers that to be the root until the next cold boot.  All 81 files
> for that particular wiki and language would be in that subdirectory,
> including the big wiki data files and the fonts and the remaining files
> (45 files, only 381KB, and this includes ALL of the executables!)  While
> the single app is running, it would not have to access (or even know
> about) anything outside its current directory, so no filesystem calls
> relating to directory navigation would be needed within that particular
> kernel.elf.  Only the initial wiki selection app (we would have to write
> one) would have to understand subdirectories, and only to one level deep.

Yeah that should work.

  -Sean

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[EdorFaus]

On Saturday 31 October 2009 02:35:42 Sean Moss-Pultz wrote:
> On Sat, Oct 31, 2009 at 3:46 AM, Doug Jones <[hidden email]> wrote:
> > (1) Has OpenMoko made the policy decision that filenames will be limited
> > to 8.3?
>
> We're using FAT.

I have a usable grasp of how FAT works, so I figured this would be the answer
while reading this thread, and thought I'd reply with an explanation even
before I saw your reply. It's not just policy, there are technical reasons.

While I suppose it's really enough to accept that this is how it is, I guess
people with less of a grasp on how FAT works might need an explanation to
really understand why this is the only sensible choice, so here goes:

The FAT filesystem inherently supports up to 8.3 filenames, and nothing more.
This is a limitation of the on-disk format used for directories (including the
root directory).

Long file name (LFN) support has been implemented on top of this (aka VFAT) in
some systems, but it is really a hack: it adds extra directory entries that
are (iirc) marked as being deleted files, and then uses the rest of the data
(both filename and other fields, like size) for those entries to contain
characters representing the long filename for the following directory entry
(the real file). Thus, implementing this is a bit complex and hacky.

This is why DOS (and the DOS mode of W9x) doesn't see the long file names (just
foobar~1.ext), and might destroy the LFN if you do certain things with the
files (such as moving to another dir) - they were made before LFNs existed.

Besides, even with just 8.3, that's 11 bytes that can be chosen pretty much
arbitrarily (only a few values you can't use); for most purposes you don't
need even that, and can put conventions into place (such as using a specific
.ext for specific file types).

Personally, I see no reason why the wikireader should need more than 8.3
filenames, at least for the kernel to know what app to start, or the reader app
to find its data files.


> > (2) How complicated will it be to implement subdirectory support?
>
> Not a problem. We're going to do something like this when we support
> more languages.

That certainly seems like both the easiest and most sensible approach.


> > Note that only one subdirectory level is really needed to implement what
> > has already been suggested.

This is kind of interesting in a way. The first DOS versions (that supported
directories at all) only supported one level. :)


> > We could adopt a brain-swap approach:  After bootup, the user selects
> > one wiki and then the app switches to the selected subdirectory and
> > considers that to be the root until the next cold boot.  All 81 files
<snip>
> > kernel.elf.  Only the initial wiki selection app (we would have to write
> > one) would have to understand subdirectories, and only to one level deep.
>
> Yeah that should work.

I thought of the same, and afaik there's just one small gotcha (not really a
problem though): subdirectories are not quite the same on-disk as the root
directory.

The format of the data in the directories is the same, sure. But
subdirectories are stored the same way files are - in a chain of clusters -
while the root directory is a specific area on the disk, determined at FS
creation time. This is why the root directory has a limited number of files it
can hold, while subdirectories don't.

So, the code reading the directory entries from the disk still has to know if
it's reading the root dir or a subdirectory, to know where on the disk to load
the entries from. The code parsing those entries can be the same though.

One way to simplify this could be to *always* keep apps such as the wikireader
in a subdirectory, never in the root directory, as then only the app selector
needs to know how to read the root dir.

Of course, if the filesystem reading/parsing code is provided by the kernel, as
a service to the programs, it could take care of this (rather small) difference
under the hood, without each app having to deal with it on its own - that
seems much more sensible to me, since it needs to know how to read FAT anyway.


That could also open up another intriguing possibility... Copying the app
selector into a subdir that has subdirs of its own. From the POV of any app
that doesn't understand directories, it doesn't really matter how deep it is,
and even if it does understand them, it still doesn't matter as long as the
current directory is considered the root and the .. directory is ignored.

Then, if the selector was implemented so that it could switch root to the ..
directory and (re)start that app, you could even move up again in the tree...

Well, this all assumes the "root" subdir selection is done by keeping the
cluster number or some such (as opposed to keeping the path by name).


--
Frode Austvik

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Christopher Hall]

On Fri, 30 Oct 2009 12:46:57 -0700
Doug Jones <[hidden email]> wrote:

> [snip]
> > I like this approach, just remind that as far I see the code and by
> > comments of the devs, the kernel implements the bare just enough to
> > read files, so I think directories are not implemented at all that's
> > why all is on root directory so at least basic hierarchical
> > filesystem has to be implemented before we can do this solution.
> > But directories will easy the organization of pictures too
>
>
> Good point.
>
> So two important questions to be answered, before we get any further
> into this:
>
> (1) Has OpenMoko made the policy decision that filenames will be
> limited to 8.3?

I would prefer to keep it simple, the boot loader must run in under 7kB
of internal RAM (actually it uses overlays) so adding more code
here is not so easy.  
>
>
> (2) How complicated will it be to implement subdirectory support?
The version of the file system supports directory access, but there
is no support for "chdir"

we are using this:   http://elm-chan.org/fsw/ff/00index_e.html
presently at version: R0.06

so all pathnames have to be absolute, I tried a couple of quick
tests and I could create and read a file in the folder, but I am missing
"mkdir" in forth, (just need to add the interface routine)

It looks like there is a new version with "chdir" support, but
I have not investigated this yet

Hope this answers your questions

>
>
> Note that only one subdirectory level is really needed to implement
> what has already been suggested.
>
> The current implementation contains 81 files, totaling 4.2GB for the
> English version.  Nearly all of that is in the big wiki data files
> (pedia*).  The other files, the ones you get when you make install,
> comprise 49 files and only 18MB, and most of that is fonts (which are
> often different for different languages).
>
> We could adopt a brain-swap approach:  After bootup, the user selects
> one wiki and then the app switches to the selected subdirectory and
> considers that to be the root until the next cold boot.  All 81 files
> for that particular wiki and language would be in that subdirectory,
> including the big wiki data files and the fonts and the remaining
> files (45 files, only 381KB, and this includes ALL of the
> executables!)  While the single app is running, it would not have to
> access (or even know about) anything outside its current directory,
> so no filesystem calls relating to directory navigation would be
> needed within that particular kernel.elf.  Only the initial wiki
> selection app (we would have to write one) would have to understand
> subdirectories, and only to one level deep.
>
>
> _______________________________________________
> Openmoko community mailing list
> [hidden email]
> http://lists.openmoko.org/mailman/listinfo/community

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community

[Doug Jones]

Christopher Hall wrote:

> On Fri, 30 Oct 2009 12:46:57 -0700
> Doug Jones <[hidden email]> wrote:
>
>> [snip]
>>> I like this approach, just remind that as far I see the code and by
>>> comments of the devs, the kernel implements the bare just enough to
>>> read files, so I think directories are not implemented at all that's
>>> why all is on root directory so at least basic hierarchical
>>> filesystem has to be implemented before we can do this solution.
>>> But directories will easy the organization of pictures too
>>
>> Good point.
>>
>> So two important questions to be answered, before we get any further
>> into this:
>>
>> (1) Has OpenMoko made the policy decision that filenames will be
>> limited to 8.3?
>
> I would prefer to keep it simple, the boot loader must run in under 7kB
> of internal RAM (actually it uses overlays) so adding more code
> here is not so easy.  
>>
>> (2) How complicated will it be to implement subdirectory support?
> The version of the file system supports directory access, but there
> is no support for "chdir"
>
> we are using this:   http://elm-chan.org/fsw/ff/00index_e.html
> presently at version: R0.06
>
> so all pathnames have to be absolute, I tried a couple of quick
> tests and I could create and read a file in the folder, but I am missing
> "mkdir" in forth, (just need to add the interface routine)
>
> It looks like there is a new version with "chdir" support, but
> I have not investigated this yet
>
> Hope this answers your questions
>

Yes, thanks for that.  Helps a lot.


This is what I've got in mind:


No changes to boot loader.  The only things to be changed are on the SD
card.

Suppose the user wants two different wikis on the device.  To keep
things simple, each wiki is independent and resides in its own
directory.  Once a particular wiki is running, there is no way to access
the other one without rebooting.  For backwards compatibility, we let
the default wiki reside at root, just as in the current implementation.

A directory may contain a Wikipedia in a different language, or perhaps
some entirely different wiki using the same WikiReader app, or perhaps
some other app that is yet to be written.  (For convenience, we call
each one an app here, even if the subdirectory actually contains
multiple applications).


The boot loader runs /kernel.elf as before, but instead of the
WikiReader app, it's just a menu system (the WikiReader app has been
renamed something like wrkernel.elf).  It scans the subdirectories (only
one level down) on the SD card, looking for a particular filename, say
menu.png, which contains an image of the menu entry for that particular
app.  By using pictures for the entries, we don't need any
(multi-language!) font support in the menu app, kernel.elf.

If it finds no /*/menu.png then it loads /wrkernel.elf and the device
works just as it does now, with everything at / on the card.  The user
does not see anything different from what they see now.

If it does find some /*/menu.png, it displays a menu containing all of
the bitmaps found (including the default one at /menu.png).  If the user
selects one other than the default, say for example the one at
/pt/menu.png, kernel.elf passes control to /pt/wrkernel.elf and the
Português version of Wikipedia runs.  From that point on, the app
doesn't need to know anything about what's at / or any other
subdirectory.  All of the .bmf font files and Forth files and whatnot
are in that subdirectory.

So kernel.elf needs to be able to do dir /*/menu.png, wrkernel.elf needs
to be able to do chdir (once), and nobody needs to do mkdir.  All other
filesystem operations happen within a single directory.


This initial menu system also is displayed if the user holds down a
button while booting, so /pt/forth.elf or /pt/calc.elf or whatever will
be run as appropriate.


With this scheme, the 8.3 filename limit is no problem.  And different
language versions of Wikipedia can reside in directories named after the
language designations used by Wikipedia.

If a user wants to add a wiki of their own, they just use the existing
tools to build the pedia* files from an xml dump.  They put these into a
folder along with the other files copied from / on a standard WikiReader
SD card, add a menu.png file for the menu entry (we can give them a tool
to generate one in their language), and then they copy the folder onto
the SD card and put it into the device and off they go.

If somebody writes an entirely different app, all they have to do is
name it wrkernel.elf, supply a menu.png, put it all in a folder and do
as above.


And nobody has to flash the ROM.

>>
>> Note that only one subdirectory level is really needed to implement
>> what has already been suggested.
>>
>> The current implementation contains 81 files, totaling 4.2GB for the
>> English version.  Nearly all of that is in the big wiki data files
>> (pedia*).  The other files, the ones you get when you make install,
>> comprise 49 files and only 18MB, and most of that is fonts (which are
>> often different for different languages).
>>
>> We could adopt a brain-swap approach:  After bootup, the user selects
>> one wiki and then the app switches to the selected subdirectory and
>> considers that to be the root until the next cold boot.  All 81 files
>> for that particular wiki and language would be in that subdirectory,
>> including the big wiki data files and the fonts and the remaining
>> files (45 files, only 381KB, and this includes ALL of the
>> executables!)  While the single app is running, it would not have to
>> access (or even know about) anything outside its current directory,
>> so no filesystem calls relating to directory navigation would be
>> needed within that particular kernel.elf.  Only the initial wiki
>> selection app (we would have to write one) would have to understand
>> subdirectories, and only to one level deep.
>>
>>
>> _______________________________________________
>> Openmoko community mailing list
>> [hidden email]
>> http://lists.openmoko.org/mailman/listinfo/community
>
> _______________________________________________
> Openmoko community mailing list
> [hidden email]
> http://lists.openmoko.org/mailman/listinfo/community
>

_______________________________________________
Openmoko community mailing list
[hidden email]
http://lists.openmoko.org/mailman/listinfo/community