Meta Tiled paletted images - PIL issue

[Ivan Mincik-2]

I was looking for the reason, why we get corrupted image using
Tilecache meta tiling with paletted images. (QUANTIZE_FORCE=ON,
QUANTIZE_DITHER=OFF, QUANTIZE_COLORS=256 in UMN Mapserver).

For others experiencing this problem I would like to note, that it
could be a PIL issue in processing paletted images.
I have tried following simple code (just opening and saving the image):

#!/usr/bin/python
import Image

im = Image.open('test.png')
print 'Image mode: ', im.mode
print 'Info: ', im.info
im.save('out.png', transparency=0)

Output image is different than the original image. Why ? PIL seams to
be loosing transparency value which is different for each color in PNG
image.
For example: 'test.png' is containing color 111 111 111 44 in palette.
After opening and saving in PIL we get 111 111 111 255.
I was trying also to extract palette from original image by using
getpalette() function and put it back with putpalette(), but the
result was the same.

So, I think, we can not use meta tiling in paletted images because of
PIL limits.

For anybody wanting to look at this problem, You can download my test
image here:
http://gista.sk/dl/test.png

Ivan



_______________________________________________
Tilecache mailing list
[hidden email]
http://openlayers.org/mailman/listinfo/tilecache

[Travis Kirstine]

You can try getting the transparency value from the im.info dictionary

im.save('out.png', 'PNG', transparency=im.info['transparency'])

Regards

2009/10/19 Ivan Mincik <[hidden email]>:

>
> I was looking for the reason, why we get corrupted image using
> Tilecache meta tiling with paletted images. (QUANTIZE_FORCE=ON,
> QUANTIZE_DITHER=OFF, QUANTIZE_COLORS=256 in UMN Mapserver).
>
> For others experiencing this problem I would like to note, that it
> could be a PIL issue in processing paletted images.
> I have tried following simple code (just opening and saving the image):
>
> #!/usr/bin/python
> import Image
>
> im = Image.open('test.png')
> print 'Image mode: ', im.mode
> print 'Info: ', im.info
> im.save('out.png', transparency=0)
>
> Output image is different than the original image. Why ? PIL seams to
> be loosing transparency value which is different for each color in PNG
> image.
> For example: 'test.png' is containing color 111 111 111 44 in palette.
> After opening and saving in PIL we get 111 111 111 255.
> I was trying also to extract palette from original image by using
> getpalette() function and put it back with putpalette(), but the
> result was the same.
>
> So, I think, we can not use meta tiling in paletted images because of
> PIL limits.
>
> For anybody wanting to look at this problem, You can download my test
> image here:
> http://gista.sk/dl/test.png
>
> Ivan
>
>
> _______________________________________________
> Tilecache mailing list
> [hidden email]
> http://openlayers.org/mailman/listinfo/tilecache
>
>

--
Travis K.

Toronto, Canada
------------------------------------------------------------
"She knows there's no success like failure
and that failure's no success at all."
-Bob Dylan-
------------------------------------------------------------
_______________________________________________
Tilecache mailing list
[hidden email]
http://openlayers.org/mailman/listinfo/tilecache

[Ivan Mincik-2]

On Mon, Oct 19, 2009 at 2:56 PM, Travis Kirstine
<[hidden email]> wrote:
> You can try getting the transparency value from the im.info dictionary
>
> im.save('out.png', 'PNG', transparency=im.info['transparency'])

Thanks for reply, but this is what I have already tried. 'im.info' is
containing just this: {'transparency': 0, 'Software': 'UMN
Mapserver'}.
I need transparency value for every palette color.

>
> Regards
>
> 2009/10/19 Ivan Mincik <[hidden email]>:
>>
>> I was looking for the reason, why we get corrupted image using
>> Tilecache meta tiling with paletted images. (QUANTIZE_FORCE=ON,
>> QUANTIZE_DITHER=OFF, QUANTIZE_COLORS=256 in UMN Mapserver).
>>
>> For others experiencing this problem I would like to note, that it
>> could be a PIL issue in processing paletted images.
>> I have tried following simple code (just opening and saving the image):
>>
>> #!/usr/bin/python
>> import Image
>>
>> im = Image.open('test.png')
>> print 'Image mode: ', im.mode
>> print 'Info: ', im.info
>> im.save('out.png', transparency=0)
>>
>> Output image is different than the original image. Why ? PIL seams to
>> be loosing transparency value which is different for each color in PNG
>> image.
>> For example: 'test.png' is containing color 111 111 111 44 in palette.
>> After opening and saving in PIL we get 111 111 111 255.
>> I was trying also to extract palette from original image by using
>> getpalette() function and put it back with putpalette(), but the
>> result was the same.
>>
>> So, I think, we can not use meta tiling in paletted images because of
>> PIL limits.
>>
>> For anybody wanting to look at this problem, You can download my test
>> image here:
>> http://gista.sk/dl/test.png
>>
>> Ivan
>>
>>
>> _______________________________________________
>> Tilecache mailing list
>> [hidden email]
>> http://openlayers.org/mailman/listinfo/tilecache
>>
>>
>
>
>
> --
> Travis K.
>
> Toronto, Canada
> ------------------------------------------------------------
> "She knows there's no success like failure
> and that failure's no success at all."
> -Bob Dylan-
> ------------------------------------------------------------
>

_______________________________________________
Tilecache mailing list
[hidden email]
http://openlayers.org/mailman/listinfo/tilecache

[Ivan Mincik-2]

On Friday 23 October 2009, you wrote:
> I don't think many programs/libraries work with paletted transparency in
> PNGs. Nor does PIL. So I'm just writing to confirm that this is a missing
> feature of PIL.
> Tibor

No solution found also after writing to PIL mailing list, so we can not use nice optimized images produced by UMN Mapserver.
We decided to produce RGBA images from Mapserver and write a "patch"  to Tilecache to optimize images just after creation using external tools like pngnq of optipng.

Ivan

>
> On Mon, Oct 19, 2009 at 8:14 PM, Ivan Mincik <[hidden email]> wrote:
>
> > On Mon, Oct 19, 2009 at 2:56 PM, Travis Kirstine
> > <[hidden email]> wrote:
> > > You can try getting the transparency value from the im.info dictionary
> > >
> > > im.save('out.png', 'PNG', transparency=im.info['transparency'])
> >
> > Thanks for reply, but this is what I have already tried. 'im.info' is
> > containing just this: {'transparency': 0, 'Software': 'UMN
> > Mapserver'}.
> > I need transparency value for every palette color.
> >
> > >
> > > Regards
> > >
> > > 2009/10/19 Ivan Mincik <[hidden email]>:
> > >>
> > >> I was looking for the reason, why we get corrupted image using
> > >> Tilecache meta tiling with paletted images. (QUANTIZE_FORCE=ON,
> > >> QUANTIZE_DITHER=OFF, QUANTIZE_COLORS=256 in UMN Mapserver).
> > >>
> > >> For others experiencing this problem I would like to note, that it
> > >> could be a PIL issue in processing paletted images.
> > >> I have tried following simple code (just opening and saving the image):
> > >>
> > >> #!/usr/bin/python
> > >> import Image
> > >>
> > >> im = Image.open('test.png')
> > >> print 'Image mode: ', im.mode
> > >> print 'Info: ', im.info
> > >> im.save('out.png', transparency=0)
> > >>
> > >> Output image is different than the original image. Why ? PIL seams to
> > >> be loosing transparency value which is different for each color in PNG
> > >> image.
> > >> For example: 'test.png' is containing color 111 111 111 44 in palette.
> > >> After opening and saving in PIL we get 111 111 111 255.
> > >> I was trying also to extract palette from original image by using
> > >> getpalette() function and put it back with putpalette(), but the
> > >> result was the same.
> > >>
> > >> So, I think, we can not use meta tiling in paletted images because of
> > >> PIL limits.
> > >>
> > >> For anybody wanting to look at this problem, You can download my test
> > >> image here:
> > >> http://gista.sk/dl/test.png
> > >>
> > >> Ivan
> > >>
> > >>
> > >> _______________________________________________
> > >> Tilecache mailing list
> > >> [hidden email]
> > >> http://openlayers.org/mailman/listinfo/tilecache
> > >>
> > >>
> > >
> > >
> > >
> > > --
> > > Travis K.
> > >
> > > Toronto, Canada
> > > ------------------------------------------------------------
> > > "She knows there's no success like failure
> > > and that failure's no success at all."
> > > -Bob Dylan-
> > > ------------------------------------------------------------
> > >
> > _______________________________________________
> > Tilecache mailing list
> > [hidden email]
> > http://openlayers.org/mailman/listinfo/tilecache
> >
>

--
Ivan Mincik
Gista s.r.o.


_______________________________________________
Tilecache mailing list
[hidden email]
http://openlayers.org/mailman/listinfo/tilecache

[soatley]

Ivan Mincik-2 wrote:

On Friday 23 October 2009, you wrote:
> I don't think many programs/libraries work with paletted transparency in
> PNGs. Nor does PIL. So I'm just writing to confirm that this is a missing
> feature of PIL.
> Tibor

No solution found also after writing to PIL mailing list, so we can not use nice optimized images produced by UMN Mapserver.
We decided to produce RGBA images from Mapserver and write a "patch"  to Tilecache to optimize images just after creation using external tools like pngnq of optipng.

Ivan

We use OptiPNG here and it works pretty good, but unfortunately, it does take a fair amount of time to run.  On some of our larger layers (ortho) it saves a lot of space.

Shawn

[Ivan Mincik-2]

On Friday 23 October 2009, soatley wrote:

>
> Ivan Mincik-2 wrote:
> >
> > On Friday 23 October 2009, you wrote:
> >> I don't think many programs/libraries work with paletted transparency in
> >> PNGs. Nor does PIL. So I'm just writing to confirm that this is a missing
> >> feature of PIL.
> >> Tibor
> >
> > No solution found also after writing to PIL mailing list, so we can not
> > use nice optimized images produced by UMN Mapserver.
> > We decided to produce RGBA images from Mapserver and write a "patch"  to
> > Tilecache to optimize images just after creation using external tools like
> > pngnq of optipng.
> >
> > Ivan
> >
>
> We use OptiPNG here and it works pretty good, but unfortunately, it does
> take a fair amount of time to run.  On some of our larger layers (ortho) it
> saves a lot of space.
>
> Shawn
>

Since now we used pngnq for postprocessing of Tilecache disk cache. But, in case of constant cache updates containing many milions of images, we have found a pain in detection, which image is already optimized, which one not. Therefore I think, it should be better to process every image just after saving to disk.

Ivan


_______________________________________________
Tilecache mailing list
[hidden email]
http://openlayers.org/mailman/listinfo/tilecache

[Mark Deneen-2]

I'm sorry if I may have missed something, but this is how I currently "optimize" png files w/TileCache:


My url in tilecache.cfg goes to a "quantize" wsgi handler, which is written in Python and PythonMagick


#!/usr/local/bin/python2.6

import PythonMagick
import urllib

def application(environ, start_response):
        status = '200 OK'
        query = environ["REQUEST_URI"][10:]
        data = urllib.urlopen("http://www.example.com/mapserv?%s" %query)
        blob = PythonMagick.Blob(data.read())
        image = PythonMagick.Image(blob)
        image.quantizeColors(64)
        image.quantizeDither(False)
        image.quantize()
        image.write(blob)
        response_headers = [('Content-type', 'image/png'), ('Content-Length', str(blob.length()))]
        start_response(status, response_headers)
        return [blob.data]

if __name__ == '__main__':
    from flup.server.fcgi import WSGIServer
    WSGIServer(application).run()


This results in an 8 bit png file with 64 colors.  It's not perfect; the color map from PNG to PNG may be different, depending on the other colors in the image.  My next step is to calculate a global color map and quantize to those colors only.  I don't believe this will be terribly difficult.

So, TileCache calls quantize which calls (in our setup) mapserv with the appropriate parameters.  The image returned from mapserv is quantized to an 8 bit image with 64 colors

file ./04/000/000/040/000/000/125.png
./04/000/000/040/000/000/125.png: PNG image data, 256 x 256, 8-bit colormap, non-interlaced


Best Regards,
Mark Deneen
Saucon Technologies

----- "Ivan Mincik" <[hidden email]> wrote:

> On Friday 23 October 2009, you wrote:
> > I don't think many programs/libraries work with paletted
> transparency in
> > PNGs. Nor does PIL. So I'm just writing to confirm that this is a
> missing
> > feature of PIL.
> > Tibor
>
> No solution found also after writing to PIL mailing list, so we can
> not use nice optimized images produced by UMN Mapserver.
> We decided to produce RGBA images from Mapserver and write a "patch"
> to Tilecache to optimize images just after creation using external
> tools like pngnq of optipng.
>

_______________________________________________
Tilecache mailing list
[hidden email]
http://openlayers.org/mailman/listinfo/tilecache