My fcgi wrapper is giving me a hard one...

[Kevin Ingwersen (Ingwie Phoenix)]

Hey everyone.

I had forked a project that promised to let me run PHP scripts trough fcgi and did slight modifications to it so it would work properly with the Yii framework. But now I just tried to print an image verbatim - changing the content type and and adding Content-length to the headers. What I had to learn was, that my output was off. I compared it to running the same script from NGINX and compared the output. It appears that the first bunch of bites are off entirely. x.x

Since I am not quite a FCGI-nut, I can not seem to fix this.

My fork: https://github.com/IngwiePhoenix/connect-yii

And here is my test:

Ing...@Ingwies-Macbook-Pro.local ~ $ cat public_html/blah.php

<?php

$file = "/Users/Ingwie/Downloads/xynu_icon.jpg";

header("Content-type: image/jpeg; charset=binary");

header("Content-length: ".filesize($file));

@readfile($file);

Ing...@Ingwies-Macbook-Pro.local ~ $ fstat Downloads/xynu_icon.jpg

-bash: fstat: command not found

Ing...@Ingwies-Macbook-Pro.local ~ $ stat Downloads/xynu_icon.jpg

16777221 15545045 -rw-r--r-- 1 Ingwie staff 0 250937 "Jan 13 01:02:42 2015" "Dec 29 18:59:25 2014" "Dec 29 18:59:25 2014" "Dec 29 18:58:54 2014" 4096 496 0 Downloads/xynu_icon.jpg

Ing...@Ingwies-Macbook-Pro.local ~ $ curl -v http://localhost:8080/blah.php > test.jpeg

* Hostname was NOT found in DNS cache

*   Trying 127.0.0.1...

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to localhost (127.0.0.1) port 8080 (#0)

> GET /blah.php HTTP/1.1

> User-Agent: curl/7.38.0

> Host: localhost:8080

> Accept: */*

>

< HTTP/1.1 200 OK

< X-Powered-By: PHP/5.5.14

< Content-type: image/jpeg; charset=binary

< Content-length: 250937

< X-Response-Time: 1.924ms

< Date: Tue, 13 Jan 2015 00:08:59 GMT

< Connection: keep-alive

<

{ [data not shown]

100  245k  100  245k    0     0  11.3M      0 --:--:-- --:--:-- --:--:-- 12.5M

* Connection #0 to host localhost left intact

Ing...@Ingwies-Macbook-Pro.local ~ $ stat test.jpeg

16777221 17084144 -rw-r--r-- 1 Ingwie staff 0 250937 "Jan 13 01:09:00 2015" "Jan 13 01:08:59 2015" "Jan 13 01:08:59 2015" "Jan 13 00:52:55 2015" 4096 496 0 test.jpeg

Ing...@Ingwies-Macbook-Pro.local ~ $ cmp test.jpeg Downloads/xynu_icon.jpg

test.jpeg Downloads/xynu_icon.jpg differ: char 1, line 1

Ing...@Ingwies-Macbook-Pro.local ~ $ hexdump -C -n 16 Downloads/xynu_icon.jpg

00000000  ff d8 ff e0 00 10 4a 46  49 46 00 01 01 01 01 2c  |......JFIF.....,|

00000010

Ing...@Ingwies-Macbook-Pro.local ~ $ hexdump -C -n 16 test.jpeg

00000000  ef bf bd ef bf bd ef bf  bd ef bf bd 00 10 4a 46  |..............JF|

00000010

As you can see at the hexdumps at the end, something is not right with the output.

Is there anyone who could help me solve this?

Kind regards,

Ingwie.

[Ryan Schmidt]

"EF BF BD" is the UTF-8 representation of U+FFFD, the Unicode replacement character.

http://en.http://en.wikipedia.org/wiki/Specials_(Unicode_block)#Replacement_character

So it seems that something is assuming that the incoming binary data is properly UTF-8 encoded, which it of course isn't, and so it replaces each of the bytes that are not valid UTF-8 characters (in your example, "FF", "D8", "FF", "E0") with the UTF-8 representation of the replacement character.

[Kevin Ingwersen (Ingwie Phoenix)]

Interesting. I haven’t even thought about UTF-8 issues before. I did use utf8 to dump some console output, but looking closer, I saw one of these sneaking into a write.

After a while I was unsatisfied with the structure, and re-wrote the php example from within the fastcgi-parser module and placed that one in. It works, the output is as it should be and I made sure that the UTF8-ness doesn’t strike me again. :)

Thank you very much! This taught me a lesson about encoding today.

A lesson a day, keeps the teacher away….sadly it doesnt, haha.

Kind regards, Ingwie!