Hi everyone,
Hi João,
I need some advice on what I might be doing wrong. I have been using the dlink firmware with ffp 0.7 and optware on an DNS-320L
for a while now. Currently I am giving Alt-F a shot. I flashed "Alt-F-0.1RC4-DNS-320L-rev-A1.bin" and have applied the "Fixes for the DNS-320L" and the Hotfix "006-DNS320L_support_sysctrl_webUI".
As far as I can tell the "basic" operation works fine (transfer speeds w/o dlink-bloatware, yey!) and I want to thank you for the effort you have put into this!
But whenever I try to complete my previous nas-setup with a multimedia-chain
(sickbeard, couchpotato, nzbget, transmission, minidlna, sicksubs)
I end up obviously breaking the firmware by some steps I take.
The problem I run into is, that after doing some modification (see below) and doing a reboot (!) I can't login to the web-interface - "Use only ASCII characters for the password, no spaces: [...]". I deleted etc/web-secret as suggested in another post
- same error, when the webui asks for a new password. I am able to regain access by removing the harddrive(s), boot from a usb and then remove/rename the Alt-F folder on the (1st) harddrive after hotplugging.
However I don't think it's a bug necessarily, rather I am probably doing something wrong which breaks the password verification process. On the other hand the only obvious thing that is broken is the login to the webui. I still can ssh and all above mentioned services seem (!) to just work fine.
These are the steps I take to setup my chain:
- do basic configurations on webinterface
- install packages from webinterface
- access via ssh
- aufs.sh -n before doing changes in /mnt/sda2/Alt-F as stated in "Customize Firmware" (wiki)
- set ipkg flag hold
aufs on / type aufs (rw,relatime,si=b3db1b7d,udba=notify)
for sickbeard and couchpotato and replace them in /opt/ with a copy from my previous ffp-setup. Yes, this might be a bit special, but is needed because I am on the development branches - and my databases are incompatible with the stable branches. Plus it's python and shouldn't be a problem. As far as I can observe this is not the moment when my firmware breaks.
- optionally (I didn't do this all the time) add sicksubs, periscope in /opt with prerequisites (i think its just setuptools, beautifulsoup, with pip)
- do some copying from previous setup and editing of config files for all packages from my previous setup
- start the programs one at a time via web-ui and check if the configuration is fine (accessing directly via host.ip:port, usually not via the alt-f web-ui)
- aufs.sh -r
- everything seems to work fine
- reboot
- above mentioned error
Unfortunatelly I didn't have the time yet to do a restart after every single step to locate the exact step/service breaking the firmware.
I wonder if this has to do with programs doing changes to databases and config-files under /etc and /var/lib which are not persistent afterwards or even break parts of the firmware.
On the other hand these are the directories used in the default configuration files after installing the corresponding packages from web-ui...
I am allowed to do things under /mnt/sda2/Alt-F after aufs.sh -n, am I not?
# aufs.sh -laufs on / type aufs (rw,relatime,si=b3db1b7d,udba=notify)/mnt/sda2/Alt-F=rw/rootmnt/rw=rw/rootmnt/ro=rr/rootmnt/sqimage=rr
Should I be rather doing it directly in / (/etc, /var/lib, ...)?
The programs are allowed to change files in / (/etc, /var/lib, ...), aren't they?
What am I getting wrong?
What am I doing wrong?
Thanks for you help!
Thanks a lot for your explanations on aufs - despite your warning/disclaimer in the wiki. I can see some pitfalls I might have run into. I was totally wondering how ipkg would work with the Alt-F-directory, but it being a wrapper didn't come to my mind.
I will setup everything again from scratch on a filesystem-root-based (/) approach, but watching carefully what ../Alt-F is doing.
Looking forward to it working now and will report back on the results.
On Wednesday, October 15, 2014 9:42:38 AM UTC+1, Michael Schieke wrote:
Hi everyone,
Hi João,
I need some advice on what I might be doing wrong. I have been using the dlink firmware with ffp 0.7 and optware on an DNS-320L
The RC4.1 release is now under test, to be released soon.
for a while now. Currently I am giving Alt-F a shot. I flashed "Alt-F-0.1RC4-DNS-320L-rev-A1.bin" and have applied the "Fixes for the DNS-320L" and the Hotfix "006-DNS320L_support_sysctrl_webUI".
As far as I can tell the "basic" operation works fine (transfer speeds w/o dlink-bloatware, yey!) and I want to thank you for the effort you have put into this!
But whenever I try to complete my previous nas-setup with a multimedia-chain
(sickbeard, couchpotato, nzbget, transmission, minidlna, sicksubs)
I end up obviously breaking the firmware by some steps I take.
The problem I run into is, that after doing some modification (see below) and doing a reboot (!) I can't login to the web-interface - "Use only ASCII characters for the password, no spaces: [...]". I deleted etc/web-secret as suggested in another post
Try 'cat /etc/web-secret' to see if the old password is as you expect (no newline, look at the start of the next prompt)In the event that it isn't, do a 'echo -n foobar > /etc/web-secret', where foobar will be the new password -- start using 'foobar'.But I don't believe this to be the issue.
- same error, when the webui asks for a new password. I am able to regain access by removing the harddrive(s), boot from a usb and then remove/rename the Alt-F folder on the (1st) harddrive after hotplugging.
Somehow there must exists a /Alt-F/etc/web-secret or some other file in the Alt-F dir that is incorrect and is shadowing the one in the base firmware.
However I don't think it's a bug necessarily, rather I am probably doing something wrong which breaks the password verification process. On the other hand the only obvious thing that is broken is the login to the webui. I still can ssh and all above mentioned services seem (!) to just work fine.
These are the steps I take to setup my chain:
- do basic configurations on webinterface
- install packages from webinterface
- access via ssh
- aufs.sh -n before doing changes in /mnt/sda2/Alt-F as stated in "Customize Firmware" (wiki)
- set ipkg flag hold
ah!,ipkg is a wrapper around ipkg-cl, and basically it will do an 'aufs.sh -n', call ipkg-cl and then call 'aufs.sh -r'.Thus, your 'aufs.sh -n' will be undone by the last ipkg 'aufs.sh -r'.Always use ipkg-cl after issuing aufs.sh -n until you issue aufs.sh -r. Never call ipkg-cl without first using 'aufs.sh -n'.You can know the current aufs.sh status by using 'aufs.sh -l'; if -n is in effect, you will see something like:aufs on / type aufs (rw,relatime,si=b3db1b7d,udba=notify)
for sickbeard and couchpotato and replace them in /opt/ with a copy from my previous ffp-setup. Yes, this might be a bit special, but is needed because I am on the development branches - and my databases are incompatible with the stable branches. Plus it's python and shouldn't be a problem. As far as I can observe this is not the moment when my firmware breaks.
- optionally (I didn't do this all the time) add sicksubs, periscope in /opt with prerequisites (i think its just setuptools, beautifulsoup, with pip)
- do some copying from previous setup and editing of config files for all packages from my previous setup
- start the programs one at a time via web-ui and check if the configuration is fine (accessing directly via host.ip:port, usually not via the alt-f web-ui)
- aufs.sh -r
- everything seems to work fine
- reboot
next time try logout before rebooting, to see if login works.
- above mentioned error
Unfortunatelly I didn't have the time yet to do a restart after every single step to locate the exact step/service breaking the firmware.
I wonder if this has to do with programs doing changes to databases and config-files under /etc and /var/lib which are not persistent afterwards or even break parts of the firmware.
Can't comment on this. I believe that the only places where Alt-F uses 'aufs.sh -n/-r' is within the ipkg wrapper; otherwise the normal dir hierarchy is used.
On the other hand these are the directories used in the default configuration files after installing the corresponding packages from web-ui...
I am allowed to do things under /mnt/sda2/Alt-F after aufs.sh -n, am I not?
Only if it is strictly necessary. I seldom do that.In normal operation, when you do a 'mkdir /opt/foo', a folder /Alt-F/opt/foo will be created. If you then touch /opt/foo/bar, than the file will be at /Alt-F/opt/foo/bar; if you edit /opt/foo/bar, the change will be again under /Alt-F/opt/foo/bar.
But for this to work the folder /opt has to exists first under Alt-F -- i.e., the parent folder has to exists for a file (or folder) operation to be reflected under /Alt-F.There are other rules for file creation and deleting.# aufs.sh -laufs on / type aufs (rw,relatime,si=b3db1b7d,udba=notify)/mnt/sda2/Alt-F=rw/rootmnt/rw=rw/rootmnt/ro=rr/rootmnt/sqimage=rrread the above from bottom up. Until Alt-F is aufs mounted, all changes will go to /rootmnt/rw. After Alt-F is aufs mounted, they go to /mnt/sda2/Alt-F, this is, top rw aufs branch. Subjected to the file/folder creation/change/deletion specific to aufs, which are not simple!
Should I be rather doing it directly in / (/etc, /var/lib, ...)?YES!
The programs are allowed to change files in / (/etc, /var/lib, ...), aren't they?
yes, changes/creations/deletions will go to /Alt-F if parent folders exists.What am I getting wrong?
What am I doing wrong?
Thanks for you help!
This is not a simple matter. That's why the wiki says that aufs might cause more grief than relief. But is necessary, at least the first three (bottom-up) branches.
--
You received this message because you are subscribed to a topic in the Google Groups "Alt-F" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/alt-f/e0bBhE8bVac/unsubscribe.
To unsubscribe from this group and all its topics, send an email to alt-f+un...@googlegroups.com.
Visit this group at http://groups.google.com/group/alt-f.
For more options, visit https://groups.google.com/d/optout.
The problem I run into is, that after doing some modification (see below) and doing a reboot (!) I can't login to the web-interface - "Use only ASCII characters for the password, no spaces: [...]". I deleted etc/web-secret as suggested in another post
Try 'cat /etc/web-secret' to see if the old password is as you expect (no newline, look at the start of the next prompt)In the event that it isn't, do a 'echo -n foobar > /etc/web-secret', where foobar will be the new password -- start using 'foobar'.But I don't believe this to be the issue.
same error, when the webui asks for a new password. I am able to regain access by removing the harddrive(s), boot from a usb and then remove/rename the Alt-F folder on the (1st) harddrive after hotplugging.
Somehow there must exists a /Alt-F/etc/web-secret or some other file in the Alt-F dir that is incorrect and is shadowing the one in the base firmware.
I agree that some "misled" shadowing is the reason. But it doesn't necessarily involve the file /etc/web-secret itself. Unfortunately my knowledge of aufs doesn't suffice to analyze properly. When I have some spare time I will remove groups of files from .../Alt-F (booted from usb) to locate the conflicting files.
On the other hand I'm happy for now with the shell and everything else working just fine!
Here are some more observations I made. Maybe some rings a bell with you.
- The status page of the web-ui shows the warning "When done you must save settings" - even after a fresh restart. Though it doesn't report on hover-over which files have been modified, which it usually does I think.
- As expected, booting without hdd inserted, I can login regularly. That way I will be able to upgrade the firmware (didn't find the script yet).
- Some shadowing policies don't seem consistent to me (not being an expert!). In .../Alt-F/etc are the files passwd, group and shadow. They are marked with a dash (-) at the end, which seems to prevent them from shadowing the files in /etc. These and some files in avahi/services are the only files with a dash at the end. However there are other files (for example regarding minidlna and transmission, which seem to be included in the firmware and working without ipkg), that are shadowed by corresponding files in .../Alt-F (not having a dash at the end). I can imagine the timing of the shadowing during boot is critical here. I guess it's no problem with config files, but when files are being shadowed while executed this might explain strange behaviour. But again I'm not an expert, as this might all be adressed by you and in the aufs architecture.
I'm happy anyway with what I've got - thanks for your work and help! I will report, if I can locate the issue.
Michael
The problem I run into is, that after doing some modification (see below) and doing a reboot (!) I can't login to the web-interface - "Use only ASCII characters for the password, no spaces: [...]". I deleted etc/web-secret as suggested in another postTry 'cat /etc/web-secret' to see if the old password is as you expect (no newline, look at the start of the next prompt)In the event that it isn't, do a 'echo -n foobar > /etc/web-secret', where foobar will be the new password -- start using 'foobar'.But I don't believe this to be the issue.So I did setup everything again, avoiding changes to mnt/sda2/Alt-F, but ran into the same issue.
- Tried cat... - password is as expected
- Tried echo... - new password foobar yields the same error
- Tried rm /etc/web-secret.. - web-ui asks for a new password, but doesn't accept any with the message "Use only ASCII characters for the password, no spaces: [...]"
args='passwd=foobar&from_url='from_url=''passwd='foobar'foobar='passwd'
if test -n "$(echo \"$lpass\" | tr -d \[!-~\])"; then echo "NOT EMPTY!"; fi
same error, when the webui asks for a new password. I am able to regain access by removing the harddrive(s), boot from a usb and then remove/rename the Alt-F folder on the (1st) harddrive after hotplugging.Somehow there must exists a /Alt-F/etc/web-secret or some other file in the Alt-F dir that is incorrect and is shadowing the one in the base firmware.
I agree that some "misled" shadowing is the reason. But it doesn't necessarily involve the file /etc/web-secret itself. Unfortunately my knowledge of aufs doesn't suffice to analyze properly. When I have some spare time I will remove groups of files from .../Alt-F (booted from usb) to locate the conflicting files.
On the other hand I'm happy for now with the shell and everything else working just fine!
Here are some more observations I made. Maybe some rings a bell with you.
- The status page of the web-ui shows the warning "When done you must save settings" - even after a fresh restart. Though it doesn't report on hover-over which files have been modified, which it usually does I think.
- As expected, booting without hdd inserted, I can login regularly. That way I will be able to upgrade the firmware (didn't find the script yet).
- Some shadowing policies don't seem consistent to me (not being an expert!). In .../Alt-F/etc are the files passwd, group and shadow. They are marked with a dash (-) at the end,
which seems to prevent them from shadowing the files in /etc. These and some files in avahi/services
are the only files with a dash at the end. However there are other files (for example regarding minidlna and transmission, which seem to be included in the firmware and working without ipkg), that are shadowed by corresponding files in .../Alt-F (not having a dash at the end). I can imagine the timing of the shadowing during boot is critical here. I guess it's no problem with config files, but when files are being shadowed while executed
this might explain strange behaviour. But again I'm not an expert, as this might all be adressed by you and in the aufs architecture.
I'm happy anyway with what I've got - thanks for your work and help! I will report, if I can locate the issue.
Michael
--
Of course I'm willing. Thanks for your patience.The alert box prevents copying the text from the browser. After clicking "ok" the debug-messages disappear. So I had to extract it from the debug-console. I hope I didn't corrupt the messages.
#!/bin/sh
# find shadowed files under /Alt-F, ignoring settings files# can be used to remove all firmware customization or applied fixes
aufs.sh -n
for i in $(cd /rootmnt/ro; find . -type f ); do if test -f /Alt-F/$i; then if ! grep -q ${i:1} /etc/settings; then if cmp -s /rootmnt/ro/$i /Alt-F/$i; then echo identical: $i rm /Alt-F/$i else echo differs, do a: diff /rootmnt/ro/$i /Alt-F/$i fi fi fidone
aufs.sh -r
On Friday, October 17, 2014 7:59:36 PM UTC+1, Michael Schieke wrote:Of course I'm willing. Thanks for your patience.The alert box prevents copying the text from the browser. After clicking "ok" the debug-messages disappear. So I had to extract it from the debug-console. I hope I didn't corrupt the messages.The log look OK.So the problem must be in /usr/www/cgi-bin/common.shDoes any of /Alt-F/usr/www/cgi-bin/login.cgi, /Alt-F/usr/www/cgi-bin/login_proc.cgi or /Alt-F/usr/www/cgi-bin/common.sh exists? None should.
if common.sh does not exists under /Alt-F, another possibility is if you have installed ffp or other binaries that are overriding the check_pass function in common.sh.The relevant binaries are the 'tr' and 'httpd' commands.
Or have you started any ffp script that conflicts with Alt-F! Yes, this must be it! disable all ffp start scripts and reboot.
You can also try the next script, that identifies shadowed files, removes identical and flags the modified ones.
Thanks for your additional help.And just to let you know: I think you are doing a great work with Alt-F. My previous setup was dlink+ffp+optware. It always felt like a 3-headed hydra, all interfering with each other. It was able to do the job, but it always felt ugly and slower than necessary. Even with your current "alpha" release for the DNS-320L, I'm so much more happy than before. So if some user tells the system to delete/format/whatever his hdd, it's an osi-layer-8-problem and there is not much you can do about it. The review you told us about is rude, simply wrong and therefore not worth being payed attention to.
checkpass() {
lpass=$(httpd -d "$1")
echo "$lpass" # add this line here
return 0 # add this line here
if test -n "$lpass"; then
...
checkpass() {
echo "$1" # add this line here
return 0 # add this line here
lpass=$(httpd -d "$1")
...
checkpass() {
lpass=$(httpd -d "$1")
if test -n "$lpass"; then
echo "$lpass" # add this line here
return 0 # add this line here
if test -n "$(echo \"$lpass\" | tr -d [!-~])"; then
...
On Monday, October 20, 2014 9:33:50 AM UTC+1, Michael Schieke wrote:Regarding the login issue -- given your last details, I have no idea why that happens, but also I don't know exactly what you did to "customized the firmware": just using the DNS320L fix script and tar file?, or doing something more?
Anyway, the fast cure, and I don't like to cure without having a diagnosis first, is to disable the password characters validation, i.e., to not verify if only valid characters were input.
2014-10-20 20:37 GMT+02:00 João Cardoso:On Monday, October 20, 2014 9:33:50 AM UTC+1, Michael Schieke wrote:Regarding the login issue -- given your last details, I have no idea why that happens, but also I don't know exactly what you did to "customized the firmware": just using the DNS320L fix script and tar file?, or doing something more?"Costumizing the firmware" might be a bad wording. I just used it in my first post, because I changed files in /Alt-F, which you refere to as "costumizing the firmware" in the wiki.
Yes, I did apply the fix script and tar file. Apart from this I just installed packages and configured the packages and some (basic) parts of the OS, mostly via shell.
Thanks for the quick and dirty fix. (1) works and I applied (3) for increased security as suggested by you.Anyway, the fast cure, and I don't like to cure without having a diagnosis first, is to disable the password characters validation, i.e., to not verify if only valid characters were input.I will track down the issue by reproducing all my steps and rebooting after every single one (or at least logging out of web-ui).
I hope I will find the needed time in the next days.I suspect the triggering step to be somewhere around sickbeard or couchpotato / installation or configuration or copying the database from my previous setup.
Will report back.
" configured (...) some (basic) parts of the OS, mostly via shell" might be the answer... if you notice that other webUI pages don't behave as expected, some of these configurations (SHELL. PATH. IFS or others) might be the reason -- just an educated guess.
As for the sickbeard and couchpotato packages, why don't you just install the standard packages and afterwards overwrite the installation folders with your own?
I hope I will find the needed time in the next days.I suspect the triggering step to be somewhere around sickbeard or couchpotato / installation or configuration or copying the database from my previous setup.
Can't see how that could affect the webui login.
We have already determined that the issue is in a single line in checkpass() in common.sh:
if test -n "$(echo \"$lpass\" | tr -d [!-~])"; then
some simple tests in the command line:lpass="foo bar"; if test -n "$(echo \"$lpass\" | tr -d [!-~])"; then echo INVALID; else echo VALID; fi # prints INVALIDlpass="foobar"; if test -n "$(echo \"$lpass\" | tr -d [!-~])"; then echo INVALID; else echo VALID; fi # prints VALID
'lpass' itself is the browser HTTP encoded value entered in the password entry. If you enter 'foo bar' the received value will be 'foo+bar', if you enter 'joão' the received value will be 'jo%C3%A3o'The initial 'httpd -d' in checkpass() does the reverse, transforms the browser HTTP encoded value back:
httpd -d "foo+bar" # displays 'foo bar'httpd -d "jo%C3%A3o" # displays "joão"
good hunting :-)