Next generation multilingual support

87 messages Options
Embed this post
Permalink
1 2 3 4 5
robert rottermann

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Martin Aspeli

>
>> we must find a way to do an "inplace translation" of structural elements.
>
> Perhaps. I'm just not sure I see a good solution. Separating out
> "structure" and "content" into separate items (so that folders have no
> metadata, and are never translated) is not a good idea from a usability
> perspective, and goes against where Plone seems to be headed in terms of
> content editing in general. This one use case is not so important that
> all other editing must suffer as a result. :)
>
> I think the only way to really get language fallback like this, is to
> use some kind of render-time proxying. That is, there's only one content
> object, and only one content tree (and all URLs reflect the canonical
> language). However, upon rendering a view, we pull data from some
> container (or maybe just values annotated onto the content object or
> something like that).
>
> There are lots of problems with that approach. It's been tried before
> and it's hard. But it may be possible, at least if some restrictions can
> be accepted.

I am happy with the folder as it is.
I only would like it to offer a way to support translation without changing the
site structure.

I is not hard to provide a translated title/description when rendering the object.
It is harder to provide a translated id (which is part of the url).

my way to deal with that up to now was to use "pidgin names" for folders, so the
 generated url remains somehow self explanatory for all the languages.

robert

------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Jean Jordaan

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by robert rottermann
Hi there

> I think, the only thing needed is to flag the Item as "not in your language".
> changing the UI language is very disturbing and produces lots of confusion.

Agreed again. The only language that the UI should be in is the one
that I chose, either by setting my browser preferences or by
explicitly selecting a different UI language. Whether I choose to
browse German or Dutch documents should have no effect on my UI. Same
as when I'm using Google search or a library catalogue, I want the
chosen UI language regardless of the language of the content I'm
browsing. Viewing a translation of some content shouldn't mean "oh,
and also switch my UI language".

--
jean                                              . .. .... //\\\oo///\\

------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Yuri-11

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by robert rottermann
robert rottermann ha scritto:

> Hanno Schlichting schrieb:
>  
>> On Thu, Jun 25, 2009 at 9:09 PM, robert rottermann<[hidden email]> wrote:
>>    
>>>> Understandable, but with this the usability of the whole system suffers.
>>>> We're actually trying to move away from folders as a separate type
>>>> anyway. In Plone 5, you'll probably just have "pages" that may or may
>>>> not contain sub-pages (and listings thereof).
>>>>        
>>> exactly what I mean.
>>> it is unfortunate that we do have folders.
>>>      
>> If your specific set of use-case is better served without any
>> hierarchical organization of content, then please don't use folders.
>> Just create a "large folder" and put all content into it. The
>> "containers without an id or title" are the BTree-nodes of that large
>> folder in that case - very similar to the talkback item they don't
>> have a user visible representation and are just system internals.
>>
>> How you want to organize and structure your content in this scenario
>> is up to you. You can use all the infrastructure today to do this, it
>> just happens to be a rather uncommon way of doing things.
>>
>> Hanno
>>
>>
>>    
> I see, I can not make myself understood.
> I do *not* say not use folders. of course we need them.
> what I try to say is they should *not be used for translation support*.
> like to put different translations into different folders.
>
> we must be able to translate folders *without touching the hierarchical
> structure* of a site.
> so we must be able to translate a folder *without copying it*. copying a folder
> changes the structure of a site. which, as martin amply showed, has lots of side
> effects.
>
> it can be advisable to have different folders for different languages.
> e.g a bookstore might have very different offerings for the divers languages and
> votes to have separate folder hierarchies for every language.
> *but such a decision has nothing to do with translation*
>  


 I totally agree. What we want to translate is the text, not copy an object.

 Translations should be part of the object, I think. Some frameworks
support multilingual fields, for example. So they translate (if needed)
the fields. Maybe this does not cover all, cause objects are not only a
group of fields.


> robert
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>  


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Martin Aspeli

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Jean Jordaan
Jean Jordaan wrote:
> Hi all
>
> I agree with Robert's take on this. It doesn't work to have multiple
> trees, one per language.

Doesn't work in what context, though? There clearly are a lot of
successful websites out there that have top level language folders. I
can certainly see use cases where this is very valid.

I'm not trying to shoot you down, I just want to make sure we're not
getting too one sided.

> When a folder is translated, the translation should be stored as an
> annotation or subobject of the folder.

This is certainly possible. But then how do you ensure that the user
sees the correct thing when they navigate to the folder? A redirect?
Some kind of view level magic to pull data from the annotation?

> In the case of a page, translations are siblings, and referenced by the
> 'translationOf' relationship. In the case of a folder, the reference
> could be to a child object, which is the translation. The exceptional
> handling required in the case of a folder, would be to make the
> appropriate child object appear in the place of the folder when
> rendering and editing.

Right. This isn't so easy to do properly and transparently. :)

> Plone's model of object publishing works well if you can describe your
> domain in terms of containment, but it's a problem if you're trying to
> accommodate more than one domain at once. I.e. if your containment
> structure models translations, there's no problem (/en/doc-en,
> /de/doc-de). However, site users don't care about this. They want to
> structure the site in terms of their domain (topic areas, work groups,
> regions, ...), and won't like it if you foist a structure from the
> domain of translations on them.

You're right of course, but I'm not sure that a top level language
folder necessarily forces you to choose between linguistic
classification and containment structure. One advantage is that you can
have URLs that reflect the target language, so e.g. you have
/en/news/new-logo and /no/nyheter/ny-logo. The URL also gives a clear
indication of the language, which can be helpful in some cases.

> The I18NLayer route did have merits. One merit is that there is no
> exceptional handling required. All translations are child objects of a
> container which just groups translations, and doesn't feature as Plone
> content at all. The problem is that this kind of thing (non-content
> container objects) isn't used anywhere in Plone these days (even though
> it's quite possible), so there aren't established practices as to how it
> should work. Is it worth considering at all?

I think some kind of proxying approach is worth considering, although I
know a lot of people (myself included) have tried and failed to do this
before.

The idea is that your view may have:

   tal:replace="context/title"

but under the hood, there's a __getattribute__ on context that tricks it
into returning the value of the 'title' attribute in the target
language, with possible fallback if there is no translation.

This still creates a raft of problems. What do you do with things that
are not simple attributes? How do you handle security? How do you handle
workflow? What about auxiliary content like portlets?

It also assumes that every content class has this hook and that it's
never overridden, which probably isn't a safe assumption. Therefore, you
probably need to use some kind of translation proxy. This may be quite
possible, of course. It's not much different to Zope 3 security proxies,
for example.

But even if we do support this, it's *still* a special case. Language
fallback and single-tree structuring is not going to be appropriate for
everyone, and will almost invariable involve some kind of trade-off that
others may not want.

Cheers,
Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Martin Aspeli

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Ricardo Alves-2
Hi Ricardo,

Ricardo Alves wrote:

> Martin Aspeli wrote:
>> Language fallback
>> -----------------
>>
>> This all has a pretty direct impact on language fallback, if we
>> implement this.
>>
>> With option (b) - top level language folders - language fallback gets
>> pretty difficult. The only way to do it that I can see would be to
>> create translation for every (published?) item, all the time. This is
>> not really "fallback", but it means that you'll get a "translation" that
>> just happens to be written in the canonical language.
>
> This stub solution would bring the usability problem of content (the
> whole content area) being shown to the user in a different language than
> the selected one.

Erm... well that is language fallback, isn't it? Until it's translated,
you see it in the canonical language (or some other language for which
there is a translation that you're likely to understand).

> Well, maybe we should define what does it mean to support fallback. In
> general I think the main goal is to make all content available to users
> who surf the website in different languages, even if content is not
> available in the desired language (in this case, the canonical or the
> default language must be shown).

It can get more complex than that. For example, a Norwegian user can be
expected to prefer Danish over Swedish, and either one over German, for
example. So if an item is authored in German, language fallback may mean
that a Norwegian user sees text in German until a Swedish translation is
available, then in Swedish until a Danish translation is available, then
in Danish until a Norwegian translation is available.

This will necessarily be site policy specific and something you need to
put a lot of thought into. The only thing I think we can support out of
the box would be fallback to the canonical language.

> So it means that all listings (be it search results, folder listing,
> catalog based navigation/sitemap, etc) should include:
>
> 1) Content matching the search parameters that's in the "selected language";
>
> 2) Content matching the search parameters that don't have an available
> version/translation in the "selected language", and so the canonical
> version should be shown (or maybe the default site language if
> available). Of course, listings in this case should make it clear that
> the content is not available in the desired language (e.g. a flag next
> to the link).

Yep. You can achieve this with 'stubs' i.e. copies of the canonical
object. You want to use a secondary workflow or similar to mark it as
'untranslated and still in language X' of course.

> Then, if the user clicks the link for content in other language, it may:
>
> 1) Show content without changing the "selected language", which I
> believe would bring even more confusion.
>
> 2) Automatically change the selected language, so all UI messages will
> be in the same language as the main content. Then, ideally, if the user
> navigates away from the page, the previously selected language should be
> re-selected. Not sure if this can be done using the current negotiation
> scheme.

Actually, I disagree here. Changing the UI would be very confusing. I
think the correct solution would be to show the body text or whatever in
the fallback language and just indicate clearly which language it's in,
and why ("not yet translated: you're viewing English").

> Any of this would probably be acceptable in the context of language
> fallback, which will always be a little messy, and so in the event of
> providing fallback out-of-the-box, it should be configurable which
> behavior to have.

Right. :)

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Hanno Schlichting-4

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by robert rottermann
On Fri, Jun 26, 2009 at 7:52 AM, robert rottermann<[hidden email]> wrote:
> I am happy with the folder as it is.
> I only would like it to offer a way to support translation without changing the
> site structure.
>
> I is not hard to provide a translated title/description when rendering the object.
> It is harder to provide a translated id (which is part of the url).

Exactly. In reality you have:

http://site/finance/sales

in case of German translated to:

http://site/finanzen/vertrieb

The id is translated in the same way as the title and description. You
need to break with the notion of having the url be based on the
content id. If you want to do this, you'd better use a different
object publishing model altogether.

What you want here is a totally different way of publishing and
traversal to match urls into content space. You need to introduce one
level of indirection, where the url is first mapped to some internal
structure which then again refers to actual content objects. You can
make this work if you use the notion of a content repository, where
every object is identified by an id alone (http://site/?objid=12345)
and navigational structure and relationships are defined explicitly
elsewhere. You can think of them much more like SQL mapping tables,
where you map objects ids into urls and can do so multiple times at
different places in a url tree. In systems that work like that, the
notion of copy/paste of objects or the id of managing the site
hierarchy and the content at the same time don't work anymore.

This just is a fundamentally different model than Zope2 and Plone use
at this time.

Anyone who tried to work against the standard way of how Plone forces
you to organizes content can tell you: Don't try it. Fighting the
model inside Plone costs huge amount of time and never works that well
in the end. If you don't want the way Plone works, then use a system
which supports the way you want to organize your content.

Hanno

------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Martin Aspeli

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by robert rottermann
robert rottermann wrote:

> I see, I can not make myself understood.
> I do *not* say not use folders. of course we need them.
> what I try to say is they should *not be used for translation support*.
> like to put different translations into different folders.
>
> we must be able to translate folders *without touching the hierarchical
> structure* of a site.

Well, as Hanno pointed out, one way to do this would be to store all
your content in one big folder, but use custom traversal adapters to
build a "virtual" site structure. This would be a lot of work, though.

> so we must be able to translate a folder *without copying it*. copying a folder
> changes the structure of a site. which, as martin amply showed, has lots of side
> effects.

So does keeping everything in one place. There's no one good solution
here to solves all problems.

> it can be advisable to have different folders for different languages.
> e.g a bookstore might have very different offerings for the divers languages and
> votes to have separate folder hierarchies for every language.
> *but such a decision has nothing to do with translation*

You need to start opening your eyes to use cases other than your own, as
this is becoming a little tedious. :)

For example, take a look at http://www.norden.org.

For a lot of organisations with public facing websites, language
fallback would be a bad thing. They have editorial processes that mean
content is translated when it makes sense. That may even involve sending
things to a third party to translate in bulk (e.g. via XLIFF). They
could also have some language-specific content, so maybe 80% of content
is authored in English and translated to four other languages, and each
language has about 20% of its content custom authored.

In this type of scenario, top level folders make a lot of sense. Folder
translation also makes a lot of sense, because you can have URLs that
make sense in each language.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Martin Aspeli

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Martin Aspeli
Here's another interesting question:

Should translation support be *opt in* or *opt out*?

That is, do we say that we try to make all content translatable by
default, but you can mark some fields as language independent; or do we
say that for a type to support translation, you need to specifically
develop it to be translation aware and say which fields are translatable?

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Jean Jordaan

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Martin Aspeli
> It can get more complex than that. For example, a Norwegian user can be
> expected to prefer Danish over Swedish, and either one over German,

You can configure your languages in order of preference in your
browser. In the absence of that, fallback to canonical is fine.

--
jean                                              . .. .... //\\\oo///\\

------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Jean Jordaan

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Hanno Schlichting-4
Hi there

> Exactly. In reality you have:
>
> http://site/finance/sales
>
> in case of German translated to:
>
> http://site/finanzen/vertrieb
>
> The id is translated in the same way as the title and description. You
> need to break with the notion of having the url be based on the
> content id. If you want to do this, you'd better use a different
> object publishing model altogether.

I don't think it's quite as black and white as this.
Having different URLs for the same content is useful in the case of
SEO as well. There's at least one product which implements this for
that case:
http://projects.quintagroup.com/products/wiki/qSEOptimizer#CanonicalURL

You will still eventually land at http://site/finance/sales if the
primary language is English, but you can publish URLs like
http://site/finanzen/vertrieb .

In the I18NLayer approach, these URLs would also work (in addition to
the translated paths):
http://site/finance/sales/en
http://site/finance/sales/de
http://site/finance/sales/fr
...

> Anyone who tried to work against the standard way of how Plone forces
> you to organizes content can tell you: Don't try it. Fighting the
> model inside Plone costs huge amount of time and never works that well
> in the end.

True, but I still hope that a decent way of working with the model can be found.

--
jean                                              . .. .... //\\\oo///\\

------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Carsten Senger-3

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Martin Aspeli
Hi Martin,

--On Freitag, Juni 26, 2009 17:26:08 +0800 Martin Aspeli
<[hidden email]> wrote:

> Here's another interesting question:
>
> Should translation support be *opt in* or *opt out*?
>
> That is, do we say that we try to make all content translatable by
> default, but you can mark some fields as language independent; or do we
> say that for a type to support translation, you need to specifically
> develop it to be translation aware and say which fields are translatable?

Opt-out. If an Add-On author doesn't care about translations, others can
use his product anyway with some concessions. Being able to overwrite the
decision in a controlled way would be nice too.

..Carsten

------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Jean Jordaan

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Martin Aspeli
Hi there

>> I agree with Robert's take on this. It doesn't work to have multiple
>> trees, one per language.
>
> Doesn't work in what context, though? There clearly are a lot of
> successful websites out there that have top level language folders. I
> can certainly see use cases where this is very valid.

It doesn't work in Plone, in terms of Folders with Pages and so on. I'm
not talking about URL structures, that's another topic ..

> I'm not trying to shoot you down, I just want to make sure we're not
> getting too one sided.

Yup, appreciated.

>> When a folder is translated, the translation should be stored as an
>> annotation or subobject of the folder.
>
> This is certainly possible. But then how do you ensure that the user
> sees the correct thing when they navigate to the folder? A redirect?
> Some kind of view level magic to pull data from the annotation?

What I18NLayer did was (much shortened):

    def getObject(self):
        layer=self.Layer()
        language_tool = getToolByName(layer, 'portal_languages', None)

        for lang in self.Languages():
            if hasattr(base, lang):
                ob = getattr(layer, lang)
                return ob

I.e. it returns the appropriate child object instead of the container
for the translations.

But I'm sure there's many other ways of implementing this.

>> In the case of a page, translations are siblings, and referenced by
>> the 'translationOf' relationship. In the case of a folder, the
>> reference could be to a child object, which is the translation. The
>> exceptional handling required in the case of a folder, would be to
>> make the appropriate child object appear in the place of the folder
>> when rendering and editing.
>
> Right. This isn't so easy to do properly and transparently. :)

Yes. I'd prefer a way that requires as little exceptional handling as
possible, as mentioned in my previous message.

If it were easy, it probably would have been done ages ago ..

> You're right of course, but I'm not sure that a top level language
> folder necessarily forces you to choose between linguistic
> classification and containment structure. One advantage is that you
> can have URLs that reflect the target language, so e.g. you have
> /en/news/new-logo and /no/nyheter/ny-logo. The URL also gives a clear
> indication of the language, which can be helpful in some cases.

I'm not mad about having the language code in my face for all eternity.
I'd prefer it if http://site/news/new-logo remained possible. In the
case of I18NLayer, 'new-logo' would refer to the translation container,
which contains http://site/news/new-logo/en and
http://site/news/new-logo/no -- browsing to http://site/news/new-logo
would return the 'en' child if that is your preferred language. If you
wanted to refer someone to a specific language, you can send them
http://site/news/new-logo/en or http://site/news/new-logo/no.

For me, translated URLs are not a must-have. For example, as far as I
can tell http://www.bangkokshuho.com/servicedapartment/index.html
does not even have an English version. (As an aside, I feel that URLs
with ideograms or other non-ASCII characters in them are too painful to
mention further.)

However, translated URLs are perfectly doable (and valuable) using an
approach like the SEOptimizer I mentioned.

> I think some kind of proxying approach is worth considering, although
> I know a lot of people (myself included) have tried and failed to do
> this before.

Yes, it may turn out to be a dead end .. but I wouldn't want to write it
off out of hand.

> The idea is that your view may have:
>
>   tal:replace="context/title"
>
> but under the hood, there's a __getattribute__ on context that tricks
> it into returning the value of the 'title' attribute in the target
> language, with possible fallback if there is no translation.

In the case of I18NLayer (which I'm only harping on because I've used
it, and it's one of the fallen-by-the-wayside ideas that may deserve a
second look), 'context' may refer to two things. In the case of
http://site/news/new-logo it would refer to the translation container
proxy, which always looks up the appropriate translation and finds
'title' or whatever on it. Again, brutally shortened:

    method=I18NLayer.inheritedAttribute(name)

    if callable(method): return apply(method, args, kw)
    else: return method

In the case of http://site/news/new-logo/en it just refers to a normal
Plone object, and there's no need for trickery.

> This still creates a raft of problems. What do you do with things that
> are not simple attributes? How do you handle security? How do you
> handle workflow? What about auxiliary content like portlets?

I don't think these issues are specific to this case ..
However, I18NLayer is OK with anything that answers to getattr, e.g.
simple attributes, fields, methods. Nothing special is done for
security, 'en', 'no' and so on are Plone objects with the usual checks.
Similarly, they are workflowed individually. Retracting all translations
at once, or blocking publication until all translations have been
reviewed are legitimate use cases, but I don't think that hard with this
model.

> It also assumes that every content class has this hook and that it's
> never overridden, which probably isn't a safe assumption.

Fully agreed, I wouldn't want to make an assumption like that.

> Therefore, you probably need to use some kind of translation proxy.
> This may be quite possible, of course. It's not much different to Zope
> 3 security proxies, for example.

Yes, when I'm talking about I18NLayer, some kind of translation proxy is
exactly what I'm talking about.

> But even if we do support this, it's *still* a special case. Language
> fallback and single-tree structuring is not going to be appropriate
> for everyone, and will almost invariable involve some kind of
> trade-off that others may not want.

Quite true. I doubt any implementation will be able to cater to every
case.

This approach doesn't mandate language fallback, though. It just makes
it possible. It's true that it does not accommodate separate
per-language folder hierarchies.

I should try to say why I don't like separate per-language folder
hierarchies (in terms of Plone content. In terms of URL structure is
another story, which I also don't like but for a different reason,
mentioned above). Say we have
http://site/en/school123/sports/football/first-team and it is not yet
translated. I click on 'translate' on 'first-team', and I have to
provide translations for the parent folders first (or I have to go and
translate each of them individually first). This results in three new
folders. 'first-team' has to refer to 'erste-spann' (please forgive my
German) by UID. If I delete 'first-team', does 'erste-spann' remain?
(I.e. do I mean to delete the translation or the document?) If I'm
deleting the document, does 'schule123/sportarten/fussball/' also get
tidied away if there's no other German content? What do I do if I want
to workflow individual translations, or all of them together? If I move
'sports' to 'activities/sports', does that trigger a cascade of moves
(to 'activitaeten/sportarten', etc.)? Does the move block unless
'activities' has already been translated in all languages?

I mention these issues as none of them come up with the I18NLayer
approach (counter-issues welcome). I can translate the folders when I
get around to it, if I want to workflow or delete a translation I do so
at e.g. 'first-team/en', if I want to do them all together I do so at
'first-team'. I don't need a reference catalog to see which documents
are translations of which others. If a document doesn't have
translations, it doesn't need a proxy (i.e. 'first-team' is an English
Page until it gets translated, at which point it moves to a Page called
'en' contained within a translation container called 'first-team').

I don't like stubs with copies of the canonical language either. Having
the same content in the database 5 times for 5 languages is a drag if it
isn't really unavoidable. It's bearable for mostly-translated sites, but
for more sparsely translated sites it's not pretty.

It's true that having 'first-team' as id which shows in the final URL is
not ideal (though as I said I don't think it's all bad). Having this id
be the id of the object in the database is arguably even worse, but we
love this, don't we? ;-) It has bad side effects in other contexts as
well. For example .. a simple rename of a parent folder containing many
subfolders full of documents, triggering a recatalog from hell, oh my!

Regards,
--
jean                                              . .. .... //\\\oo///\\

------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Martin Aspeli

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
Jean Jordaan wrote:

> Hi there
>
>>> I agree with Robert's take on this. It doesn't work to have multiple
>>> trees, one per language.
>> Doesn't work in what context, though? There clearly are a lot of
>> successful websites out there that have top level language folders. I
>> can certainly see use cases where this is very valid.
>
> It doesn't work in Plone, in terms of Folders with Pages and so on. I'm
> not talking about URL structures, that's another topic ..

I don't agree with this. It certainly works for some use cases.
norden.org is a good example.

> What I18NLayer did was (much shortened):
>
>     def getObject(self):
>         layer=self.Layer()
>         language_tool = getToolByName(layer, 'portal_languages', None)
>
>         for lang in self.Languages():
>             if hasattr(base, lang):
>                 ob = getattr(layer, lang)
>                 return ob
>
> I.e. it returns the appropriate child object instead of the container
> for the translations.

When is this called?

> I'm not mad about having the language code in my face for all eternity.
> I'd prefer it if http://site/news/new-logo remained possible. In the
> case of I18NLayer, 'new-logo' would refer to the translation container,
> which contains http://site/news/new-logo/en and
> http://site/news/new-logo/no -- browsing to http://site/news/new-logo
> would return the 'en' child if that is your preferred language. If you
> wanted to refer someone to a specific language, you can send them
> http://site/news/new-logo/en or http://site/news/new-logo/no.
>
> For me, translated URLs are not a must-have. For example, as far as I
> can tell http://www.bangkokshuho.com/servicedapartment/index.html
> does not even have an English version. (As an aside, I feel that URLs
> with ideograms or other non-ASCII characters in them are too painful to
> mention further.)

Yes, but we already have a way to URL normalise.

> In the case of I18NLayer (which I'm only harping on because I've used
> it, and it's one of the fallen-by-the-wayside ideas that may deserve a
> second look), 'context' may refer to two things. In the case of
> http://site/news/new-logo it would refer to the translation container
> proxy, which always looks up the appropriate translation and finds
> 'title' or whatever on it. Again, brutally shortened:
>
>     method=I18NLayer.inheritedAttribute(name)
>
>     if callable(method): return apply(method, args, kw)
>     else: return method
>
> In the case of http://site/news/new-logo/en it just refers to a normal
> Plone object, and there's no need for trickery.

I don't know how to do this transparently and well, though.

I think the most likely solution if we want something like this is not
to treat the objects as separate, but rather to have some kind of proxy
that is applied during rendering and operates at a field level. But it'd
be fraught with difficulties anyway, and we'd need to accept certain
limitations.

> I don't think these issues are specific to this case ..
> However, I18NLayer is OK with anything that answers to getattr, e.g.
> simple attributes, fields, methods. Nothing special is done for
> security, 'en', 'no' and so on are Plone objects with the usual checks.
> Similarly, they are workflowed individually. Retracting all translations
> at once, or blocking publication until all translations have been
> reviewed are legitimate use cases, but I don't think that hard with this
> model.

No, true.

> Yes, when I'm talking about I18NLayer, some kind of translation proxy is
> exactly what I'm talking about.

Except I think we need something that's transparent. It'd probably need
to happen during traversal, so that at the last step, when you have
'context', you adapt it to ITranslatable, and that returns context
wrapped in a proxy that is transparent except that it intercepts calls
to certain attributes and looks for translated values, which are stored
elsewhere. That "elsewhere" could of course be a different content
object, but I think it would make more sense to just store them in a
separate dictionary.

This would mean that we only support translation for values stored in
attributes on the object. I think that may be a safeish assumption,
though if we are to support Archetypes we probably want something that
has awareness of its storage model.

> This approach doesn't mandate language fallback, though. It just makes
> it possible. It's true that it does not accommodate separate
> per-language folder hierarchies.

Right. And I worry that if we store everything as siblings, we'll have a
hard to managing the content (there's "four of everything" when you view
the ZMI, for example); conversely, if we store values in a dict, then
that's not very transparent.

> I should try to say why I don't like separate per-language folder
> hierarchies (in terms of Plone content. In terms of URL structure is
> another story, which I also don't like but for a different reason,
> mentioned above). Say we have
> http://site/en/school123/sports/football/first-team and it is not yet
> translated. I click on 'translate' on 'first-team', and I have to
> provide translations for the parent folders first (or I have to go and
> translate each of them individually first). This results in three new
> folders. 'first-team' has to refer to 'erste-spann' (please forgive my
> German) by UID. If I delete 'first-team', does 'erste-spann' remain?
> (I.e. do I mean to delete the translation or the document?) If I'm
> deleting the document, does 'schule123/sportarten/fussball/' also get
> tidied away if there's no other German content? What do I do if I want
> to workflow individual translations, or all of them together? If I move
> 'sports' to 'activities/sports', does that trigger a cascade of moves
> (to 'activitaeten/sportarten', etc.)? Does the move block unless
> 'activities' has already been translated in all languages?

I think there are well defined answers to all of those that make sense.
I don't want to elaborate them all here.

> I mention these issues as none of them come up with the I18NLayer
> approach (counter-issues welcome). I can translate the folders when I
> get around to it, if I want to workflow or delete a translation I do so
> at e.g. 'first-team/en', if I want to do them all together I do so at
> 'first-team'. I don't need a reference catalog to see which documents
> are translations of which others. If a document doesn't have
> translations, it doesn't need a proxy (i.e. 'first-team' is an English
> Page until it gets translated, at which point it moves to a Page called
> 'en' contained within a translation container called 'first-team').

Yes, I can see a certain appeal.

On the other hand, I am worried that have two persistent objects for
everything now. Presumably, 'school123' would be an I18N object, and
it'd have children 'en' and 'de' with the actual folder content. Then
where do sub-folders go? Under the canonical language Then you have
school123/en/sports/en and so on. And what if you end up with something
like school123/de/foo by accident?

I also wonder if this is going to break down if and when we have
folderish pages.

> I don't like stubs with copies of the canonical language either. Having
> the same content in the database 5 times for 5 languages is a drag if it
> isn't really unavoidable. It's bearable for mostly-translated sites, but
> for more sparsely translated sites it's not pretty.

Yes, definitely. If you want sparse translation and language fallback,
then that's tricky.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Martin Aspeli

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Martin Aspeli
Hi,

I've tried to summarise the options and pros and cons visually here:

   http://dist.martinaspeli.net/Multilingual.jpeg

Comments and corrections welcome!

The one option that isn't fully represented is the "I18NLayer" one. The
"proxy" option is similar, but makes different trade-offs terms of the
ease of managing translation-specific workflow, local roles, comments
and so on.

If Jean or anyone else could flesh this out, that'd be great. At the
moment, I don't really understand how it stores translations of folders
(specifically, where are children of the translated folder stored), and
how it hooks into traversal.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Ricardo Alves-2

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Martin Aspeli
Hi Martin,

Martin Aspeli wrote:

>> Well, maybe we should define what does it mean to support fallback. In
>> general I think the main goal is to make all content available to users
>> who surf the website in different languages, even if content is not
>> available in the desired language (in this case, the canonical or the
>> default language must be shown).
>
> It can get more complex than that. For example, a Norwegian user can be
> expected to prefer Danish over Swedish, and either one over German, for
> example. So if an item is authored in German, language fallback may mean
> that a Norwegian user sees text in German until a Swedish translation is
> available, then in Swedish until a Danish translation is available, then
> in Danish until a Norwegian translation is available.
>

Yes, ideally we should fallback to the language that is the best
"fallback match" for the desired language. BTW, PTS also supports
fallback when explicitly declared at the PO files. For example, the "pt"
translation for domain "plone" is registered as a fallback for "pt-mz",
"pt-ao", etc.

> This will necessarily be site policy specific and something you need to
> put a lot of thought into. The only thing I think we can support out of
> the box would be fallback to the canonical language.

And that will probably suit the large majority of fallback use cases. At
most, maybe we can have an algorithm that calculates the best fallback
sequence:

- Languages in the same family (if the user wants "pt-br", "pt" will
do). Not sure if we can have a inner-sequence in the same family. This
is a common use case for UI i18n, it won't be that common in content
translation I think.

- The default site language (it may make sense to ensure some
consistency, i.e. trying to show the user the less languages possible).

- As a last resort, the canonical language of the content.


More complex solutions (per user settings?) should be plugged-in by
other packages, I guess.


>
> Actually, I disagree here. Changing the UI would be very confusing. I
> think the correct solution would be to show the body text or whatever in
> the fallback language and just indicate clearly which language it's in,
> and why ("not yet translated: you're viewing English").

Yes, I was thinking out loud here :), but thinking a little more I agree
that this would probably bring bigger usability problems.


Ricardo

--
Ricardo Alves <[hidden email]>
Eurotux <http://www.eurotux.com>


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Ricardo Alves-2

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Yuri-11
Yuri wrote:

> robert rottermann ha scritto:
>>
>> I see, I can not make myself understood.
>> I do *not* say not use folders. of course we need them.
>> what I try to say is they should *not be used for translation support*.
>> like to put different translations into different folders.
>>
>> we must be able to translate folders *without touching the hierarchical
>> structure* of a site.
>> so we must be able to translate a folder *without copying it*. copying a folder
>> changes the structure of a site. which, as martin amply showed, has lots of side
>> effects.
>>
>> it can be advisable to have different folders for different languages.
>> e.g a bookstore might have very different offerings for the divers languages and
>> votes to have separate folder hierarchies for every language.
>> *but such a decision has nothing to do with translation*
>>  
>
>
>  I totally agree. What we want to translate is the text, not copy an object.
>
>  Translations should be part of the object, I think. Some frameworks
> support multilingual fields, for example. So they translate (if needed)
> the fields. Maybe this does not cover all, cause objects are not only a
> group of fields.
>
>

I disagree. The concept of handling translations as separated,
independent content objects is a good principle. In its essence, a
translation of a document is a new document, and a translator is also an
author, if it wasn't so, translations of the same document or book would
all be equal.


Ricardo

--
Ricardo Alves <[hidden email]>
Eurotux <http://www.eurotux.com>


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Geir Bækholt-2

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by robert rottermann
robert rottermann wrote:
> in my experience *very* little of the content of a multilingual site
> exists in all supported languages, normally not more than some percent.
> so the most important part of a multilingual solution is to provide an
> *elegant and easy to use fall-back-language support*.

Language fallback has severe usability effects, and should IMO not be
recommended in real-life use even if it might look good on paper.

Most people that come up with solutions like this are bilingual
(software developers) already, amd don't see the problem. When you
expose normal people to websites with mixed language content, their
ability to understand, browse, scan and navigate drops dramatically.

I would strongly recommend against such a feature in all but a few
special situations.

Most possible implementations of language-fall-back (while not
drastically altering how Plone works) would probably impact performance

Implementing this in the default multilingual solution will most
probably (IMHO) lead to:

- More developers making a bad usability choice
- Plone being slower in multilingual sites



--
Geir Bækholt
http://www.jarn.com


------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
jdilelle

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
I'm currently working on a site (montrealcritters.com) that requires that every bit of content is presented in two (official) languages, without excuses. It is the case for practical, socio-political and legal reasons. I wouldn't be able to tell whether other regions of the world share such requirements, but I do know that "normal" users get lost if bits of it are not translated, as it was mentioned here.

On Sat, Jun 27, 2009 at 4:29 PM, Geir Bækholt <[hidden email]> wrote:

Most people that come up with solutions like this are bilingual
(software developers) already, amd don't see the problem. When you
expose normal people to websites with mixed language content, their
ability to understand, browse, scan and navigate drops dramatically.

I would strongly recommend against such a feature in all but a few
special situations.


--
Powered by tofu

------------------------------------------------------------------------------

_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Alexander Pilz-2

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
Some javascript/style in this post has been disabled (why?)
The European Union has currently 22 official languages. Even the fact that 2/3rd of the EU staff are translators does not allow them to translate every piece of information into all languages. So while the information is complete in english, it will always be incomplete in a translation.

I agree with Geir that is is confusing for users if the language of the page suddenly switches.

But in the above case, users need to be made aware that there is more information available in another language. I think we don't want the change of language but the information that the user can switch manually, if he wants.


Juan Pablo Di Lelle schrieb:
I'm currently working on a site (montrealcritters.com) that requires that every bit of content is presented in two (official) languages, without excuses. It is the case for practical, socio-political and legal reasons. I wouldn't be able to tell whether other regions of the world share such requirements, but I do know that "normal" users get lost if bits of it are not translated, as it was mentioned here.

On Sat, Jun 27, 2009 at 4:29 PM, Geir Bækholt <[hidden email]> wrote:

Most people that come up with solutions like this are bilingual
(software developers) already, amd don't see the problem. When you
expose normal people to websites with mixed language content, their
ability to understand, browse, scan and navigate drops dramatically.

I would strongly recommend against such a feature in all but a few
special situations.


--
Powered by tofu

------------------------------------------------------------------------------

_______________________________________________ Plone-developers mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/plone-developers

Alexander Pilz

-- 
Alexander Pilz
SYSLAB.COM GmbH, Landwehrstrasse 60-62, 80336 Munich, Germany
http://www.syslab.com - Amtsgericht Muenchen, HRB 135057
Steuernummer 143/184/50154 - Ust.-ID: DE212842815

------------------------------------------------------------------------------

_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
robert rottermann

Re: Next generation multilingual support

Reply Threaded More More options
Print post
Permalink
In reply to this post by Geir Bækholt-2
Geir Bækholt schrieb:

> robert rottermann wrote:
>> in my experience *very* little of the content of a multilingual site
>> exists in all supported languages, normally not more than some percent.
>> so the most important part of a multilingual solution is to provide an
>> *elegant and easy to use fall-back-language support*.
>
> Language fallback has severe usability effects, and should IMO not be
> recommended in real-life use even if it might look good on paper.
>
> Most people that come up with solutions like this are bilingual
> (software developers) already, amd don't see the problem. When you
> expose normal people to websites with mixed language content,

this is against my experience. in any corporal intranet that spans more than one
language such dealing with more than one language is daily practice and a basic
scill to use the intranet

their
> ability to understand, browse, scan and navigate drops dramatically.

I agree that the current way LinguaPlone is handling UI when switching to a
translated document can be very confusing.
but this does not speak against a language fallback language but agains teh use
of the way LinguaPlone tackles the problem.

robert

------------------------------------------------------------------------------
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
1 2 3 4 5