why additional \r before each \r\n

[Jivanmukta]

Please explain me why this code outputs text with \r\r\n, not \r\n. I
run it in Windows 10. When I call my program with >__test.txt I see
\r\r\n in file __text.txt.
I don't know why additional \r in text outputed from get_help() - I have
single END_LINE after each line (in returned string).

#define END_LINE "\r\n"

void failure_dirtyphp(string errors) {
if (last_obfuscation_stream.is_open()) {
OUTPUT(errors);
last_obfuscation_stream.close();
} else {
cout << errors << endl;
}
unlink((wstr2str(tmp_dir()) + ::pid_filename).c_str());
exit(EXIT_FAILURE);
}

main() {
...
failure_dirtyphp(dirtyphp.get_usage() + END_LINE +
dirtyphp.get_help() + END_LINE);
...
}

string obfuscator::get_help() const {
strvector f;
vector<int> zf, yii;
for (auto it = frameworks_config.begin(); it !=
frameworks_config.end(); ++it) {
f.insert(f.end(), it->first);
if (starts_with(it->first, "zf")) {
zf.insert(zf.end(), stoi(safe_substr(it->first, 2)));
}
if (starts_with(it->first, "yii")) {
yii.insert(yii.end(), stoi(safe_substr(it->first, 3)));
}
}
sort(f.begin(), f.end());
sort(zf.begin(), zf.end());
sort(yii.begin(), yii.end());
// string config_filepath = wstr2str(installation_dir +
dir_separator + L"xml" + dir_separator) + config_filename;
return string("") +
"The program obfuscates PHP " + SUPPORTED_PHP_VERSIONS + "
application source code from php_application_dir and puts obfuscated
code into php_result_dir (so these directories must be different);
php_result_dir should not exist (it is created by dirtyphp) or it should
be empty." + END_LINE +
"Minimal required PHP version: " + min_php_version + END_LINE +
"You can obfuscate aplication using any MVC framework or using
Composer (and MVC frameworks) or you can obfuscate application written
without any framework at all." + END_LINE +
"If you don't use any framework, do not use options:
--controllers, --models, --views, (use them if you use --framework or
--vendor), --framework, --vendor and --system." + END_LINE +
"For example, if you use CodeIgniter 4, use --framework=ci4 and
--system='path to ci4 dir' and you may but you don't have to use
options: --reserved, --subdirs, --controllers, --models, --views.
Default values from xml/" + config_filename + " file will be assumed." +
END_LINE +
"Similarly use --framework=yii# for Yii version # framework (#
equal to: " + implode(" ", yii) + "), or --framework=zf# for Zend
Framework version # (# equal to: " + implode(" ", zf) + ")." + END_LINE +
"Allowed frameworks identifiers are: " + implode(", ", f) + "."
+ END_LINE +
"If you use another (not mentioned) MVC framework, use
--framework=other --system='path to framework dir' --subdirs=...
--controllers=... --models=... --views=... and optionally
--obfpctl=(0|1) --obfpmdl=(0|1) --obfpvw=(0|1) --obfvvw=(0|1)." + END_LINE +
"You may also add such framework (let's say myfrm) to xml/" +
config_filename + " configuration file and use --framework=myfrm option
(then you will not have to use --subdirs --controllers --models --views
options because configuration file contains these data)." + END_LINE +
"If you use Composer, use options --vendor='path to vendor dir'
--vendorframeworks='comma separated list of frameworks ids'
--vendorsystems='comma separated list of frameworks subdirs' instead of
--framework --system options; if --vendorframeworks contains more than 1
framework, you should use --subdirs=... --controllers=... --models=...
--views=... (they may be empty), and (not necessary) --obfpctl=(0|1)
--obfpmdl=(0|1) --obfpvw=(0|1) --obfvvw=(0|1) options." + END_LINE +
"Option --obfpctl=1 means that properties in controllers should
be obfuscated, --obfpctl=0 means that they should not." + END_LINE +
"Option --obfpmdl=1 means that properties in models should be
obfuscated, --obfpmdl=0 means that they should not." + END_LINE +
"Option --obfpvw=1 means that properties in views (if it is a
PHP class) should be obfuscated, --obfpvw=0 means that they should not."
+ END_LINE +
"Option --obfvvw=1 means that variables in views should be
obfuscated, --obfvvw=0 means that they should not." + END_LINE +
"If you use 3rd party libraries, use option --3rdparty, unless
they are in vendor folder (and you use --vendor option - in this case
you don\'t list them in --3rdparty option)." + END_LINE +
"Identifiers used by frameworks and identifiers used by 3rd
party libraries are not obfuscated." + END_LINE +
"If you use --subdirs and --framework options, all
subdirectories taken from --subdirs and xml/" + config_filename + " will
be obfuscated." + END_LINE +
"The program obfuscates *." + implode(" *.", explode(",",
default_extensions)) + " files in specified subdirectories." + END_LINE +
"Unless you use --extensions option, extensions " +
default_extensions + " are assumed." + END_LINE +
"Supply reserved variables, properties, functions (f.e. quoted
in apostrophes) and methods (f.e. _remap) in --reserved option." +
END_LINE +
"When you use --reserved option (to exclude some identifiers),
supply reserved identifiers from files only inside directories specified
in --subdirs option." + END_LINE +
"Default length of generated identifiers is " +
to_string(::default_random_identifiers_length) + " characters, use
option --idlen to change it, but not less than " +
std::to_string(::min_random_identifiers_length) + "." + END_LINE +
"Use option --maxlinelen if you want to specify maximum length
of source code lines, default is " +
std::to_string(::default_max_line_len) + "; significant increase of this
value may cause program's crash." +
#if defined(linux) || defined(__unix__) || defined(unix)
" Try: ulimit -s unlimited" +
#endif
END_LINE +
"Use option --nocheck if you want to turn off source and result
code PHP syntax checking, as well as not allowed syntax checking - but
then you have no guarantee that your application will run correctly
after obfuscation." + END_LINE +
"Use option --allowreflection if you want to allow for using
reflection (which is not allowed by default)." + END_LINE +
"Limitations: the program doesn't support (in obfuscated
application) reflection, function compact(), variable variables,
variable functions, anonymous classes. " +
"This functionality is allowed in frameworks and 3rd party
libraries. The program does not obfuscate class names nor constants
names. Single source code line may contain at last one
class/trait/interface definition. " +
"Paths used by your application (of directories and files)
should not contain comma characters nor '--' text (two dashes)." +
END_LINE +
"If you use dirtyphp from a command line, it is recommended to
prepare simple batch file or a command alias for your project to be
obfuscated." + END_LINE +
"Option --clearcache clears cache of applications, frameworks,
3rd parties, and cache of Composer's vendor folder; this operation
cannot be undone." + END_LINE;
}

--
Ta wiadomość e-mail została sprawdzona pod kątem wirusów przez oprogramowanie antywirusowe AVG.
www.avg.com

[Bo Persson]

On 2023-03-06 at 17:06, Jivanmukta wrote:
> Please explain me why this code outputs text with \r\r\n, not \r\n. I
> run it in Windows 10. When I call my program with >__test.txt I see
> \r\r\n in file __text.txt.
> I don't know why additional \r in text outputed from get_help() - I have
> single END_LINE after each line (in returned string).
>
> #define END_LINE "\r\n"
>

On Windows (and unlike Linux) the code "\n" is expanded into two
characters (Carriage Return and NewLine). This is the way it has been
since always (since mechanical printers and such were used).

[Bo Persson]

Also, this is the reason for C and C++ files having text mode and binary
mode. In binary mode there is no translation.

[Jivanmukta]

I forgot:

#define OUTPUT(x) do { last_obfuscation_stream << x; std::cout << x; }
while (false)

[Jivanmukta]

W dniu 06.03.2023 o 17:14, Bo Persson pisze:

I changed definition of END_LINE:

#define END_LINE "\n"

and it solved the problem.
Thank you for help.

[Keith Thompson]

Jivanmukta <jivan...@poczta.onet.pl> writes:
> W dniu 06.03.2023 o 17:14, Bo Persson pisze:
>> On 2023-03-06 at 17:06, Jivanmukta wrote:
>>> Please explain me why this code outputs text with \r\r\n, not
>>> \r\n. I run it in Windows 10. When I call my program with
>>> >__test.txt I see \r\r\n in file __text.txt.
>>> I don't know why additional \r in text outputed from get_help() - I
>>> have single END_LINE after each line (in returned string).
>>>
>>> #define END_LINE "\r\n"
>>>
>> On Windows (and unlike Linux) the code "\n" is expanded into two
>> characters (Carriage Return and NewLine). This is the way it has
>> been since always (since mechanical printers and such were used).
>>
> I changed definition of END_LINE:
>
> #define END_LINE "\n"
>
> and it solved the problem.
> Thank you for help.

I suggest it would be clearer to drop the END_LINE macro and use "\n"
or '\n' directly.

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for XCOM Labs
void Void(void) { Void(); } /* The recursive call of the void */

[James Kuyper]

On 3/6/23 11:06, Jivanmukta wrote:
> Please explain me why this code outputs text with \r\r\n, not \r\n. I
> run it in Windows 10. When I call my program with >__test.txt I see
> \r\r\n in file __text.txt.
> I don't know why additional \r in text outputed from get_help() - I
> have single END_LINE after each line (in returned string).
>
> #define END_LINE "\r\n"
>
> void failure_dirtyphp(string errors) {
> if (last_obfuscation_stream.is_open()) {
> OUTPUT(errors);
> last_obfuscation_stream.close();
> } else {
> cout << errors << endl;
> }
> unlink((wstr2str(tmp_dir()) + ::pid_filename).c_str());
> exit(EXIT_FAILURE);
> }
>
> main() {
> ...
> failure_dirtyphp(dirtyphp.get_usage() + END_LINE + dirtyphp.get_help()
> + END_LINE);

If you have a stream opened in text mode, when you send a '\n' to that
stream, it automatically gets converted to what method the standard
library uses to indicate the end of a line, which usually matches the
line-ending convention on that platform. The inverse conversion gets
performed when you read from a text file. As a result, you don't need to
write different code for platforms with different ways of indicating the
end of a line. See <https://en.m.wikipedia.org/wiki/Newline> for a list
of the wide variety of different methods used to represent the end of a
line in many different contexts. In particular, to break some
preconceptions you might have, take a look the descriptions for
record-based and fixed line length systems.

The C++ standard says almost nothing about text mode vs binary mode,
cross-referencing the C standard library for definitions of how the C++
I/O library routines work. Here's what the C standard says about it:

(7.21.2p2)
> A text stream is an ordered sequence of characters composed into
> lines, each line consisting of
> zero or more characters plus a terminating new-line character. Whether
> the last line requires a
> terminating new-line character is implementation-defined. Characters
> may have to be added, altered,
> or deleted on input and output to conform to differing conventions for
> representing text in the host
> environment. Thus, there need not be a one-to-one correspondence
> between the characters in a
> stream and those in the external representation. Data read in from a
> text stream will necessarily
> compare equal to the data that were earlier written out to that stream
> only if: the data consist only
> of printing characters and the control characters horizontal tab and
> new-line; no new-line character
> is immediately preceded by space characters; and the last character is
> a new-line character. Whether
> space characters that are written out immediately before a new-line
> character appear when read in
> is implementation-defined.

Note, in particular, the peculiar connection between new-lines and
spaces. That's intended to allow implementations which use fixed line
length systems, with end-of-line indicated by padding to the end with
blanks.

(7.21p3)
> Whether a write on a text stream causes
> the associated file to be truncated beyond that point is
> implementation-defined.

[Jivanmukta]

W dniu 06.03.2023 o 19:00, Keith Thompson pisze:

> Jivanmukta <jivan...@poczta.onet.pl> writes:
>> W dniu 06.03.2023 o 17:14, Bo Persson pisze:
>>> On 2023-03-06 at 17:06, Jivanmukta wrote:
>>>> Please explain me why this code outputs text with \r\r\n, not
>>>> \r\n. I run it in Windows 10. When I call my program with
>>>>> __test.txt I see \r\r\n in file __text.txt.
>>>> I don't know why additional \r in text outputed from get_help() - I
>>>> have single END_LINE after each line (in returned string).
>>>>
>>>> #define END_LINE "\r\n"
>>>>
>>> On Windows (and unlike Linux) the code "\n" is expanded into two
>>> characters (Carriage Return and NewLine). This is the way it has
>>> been since always (since mechanical printers and such were used).
>>>
>> I changed definition of END_LINE:
>>
>> #define END_LINE "\n"
>>
>> and it solved the problem.
>> Thank you for help.
>
> I suggest it would be clearer to drop the END_LINE macro and use "\n"
> or '\n' directly.
>

You are right. I used macro because I had 2 different definitions for
Linux and for Windows. Now I have identical definitions and macro is not
necessary.

[Paavo Helde]

But you still have different file contents in Windows and in Linux,
because of the "helpful" translation of line feeds in text mode. This
just complicates things and is not needed for anything because nowadays
the files are commonly shared over the network and can easily end up on
a "wrong platform".

Suggesting to always use binary mode file streams and '\n' newlines when
writing files, to get predictable and fixed behavior. Nowadays even
Notepad copes with the Unix-style newlines, so there should be no need
to have any \r\n newlines around.

[Keith Thompson]

Paavo Helde <ees...@osa.pri.ee> writes:
> 06.03.2023 20:35 Jivanmukta kirjutas:

[...]

>> You are right. I used macro because I had 2 different definitions
>> for Linux and for Windows. Now I have identical definitions and
>> macro is not necessary.
>
> But you still have different file contents in Windows and in Linux,
> because of the "helpful" translation of line feeds in text mode. This
> just complicates things and is not needed for anything because
> nowadays the files are commonly shared over the network and can easily
> end up on a "wrong platform".
>
> Suggesting to always use binary mode file streams and '\n' newlines
> when writing files, to get predictable and fixed behavior. Nowadays
> even Notepad copes with the Unix-style newlines, so there should be no
> need to have any \r\n newlines around.

Another approach is to use the native encoding for each operating
system. Many (but not all) Unix/Linux tools handle files with
Windows-style line endings.

Yes, it can be annoying, but it's workable.