blacklist shallalist

[Alessandro Peroni]

Hi,
i am using nxfilter v.3.3.4 with blacklist shallalist, i started script update_sh.sh for update but during domain test the result is always unclassified.

Why?What have i do?

Best,
Alessandro

[Jinhee]

What did it say when you run the script? Did you see any log for data update?

[Alessandro Peroni]

Hi Jinhee,
below log:
root@.........:/nxfilter/bin# ./update_sh.sh
DEBUG [07-13 09:19:32] - old_md5 = f62672a42ecc0c19c964a59f2f0a5067  shallalist.tar.gz, new_md5 = e7b6bf02d1d81db83aad958bddfe7dc5  shallalist.tar.gz
 INFO [07-13 09:19:32] - Downloading http://www.shallalist.de/Downloads/shallalist.tar.gz
It may take several minutes!
Loading domains from /nxfilter/tmp/BL/spyware/domains
Loading domains from /nxfilter/tmp/BL/remotecontrol/domains
Loading domains from /nxfilter/tmp/BL/warez/domains
Loading domains from /nxfilter/tmp/BL/webradio/domains
Loading domains from /nxfilter/tmp/BL/anonvpn/domains
Loading domains from /nxfilter/tmp/BL/dating/domains
Loading domains from /nxfilter/tmp/BL/music/domains
Loading domains from /nxfilter/tmp/BL/tracker/domains
Loading domains from /nxfilter/tmp/BL/drugs/domains
Loading domains from /nxfilter/tmp/BL/ringtones/domains
Loading domains from /nxfilter/tmp/BL/updatesites/domains
Loading domains from /nxfilter/tmp/BL/isp/domains
Loading domains from /nxfilter/tmp/BL/movies/domains
Loading domains from /nxfilter/tmp/BL/imagehosting/domains
Loading domains from /nxfilter/tmp/BL/shopping/domains
Loading domains from /nxfilter/tmp/BL/jobsearch/domains
Loading domains from /nxfilter/tmp/BL/webphone/domains
Loading domains from /nxfilter/tmp/BL/chat/domains
Loading domains from /nxfilter/tmp/BL/weapons/domains
Loading domains from /nxfilter/tmp/BL/radiotv/domains
Loading domains from /nxfilter/tmp/BL/gamble/domains
Loading domains from /nxfilter/tmp/BL/news/domains
Loading domains from /nxfilter/tmp/BL/violence/domains
Loading domains from /nxfilter/tmp/BL/religion/domains
Loading domains from /nxfilter/tmp/BL/government/domains
Loading domains from /nxfilter/tmp/BL/aggressive/domains
Loading domains from /nxfilter/tmp/BL/science/astronomy/domains
Loading domains from /nxfilter/tmp/BL/science/chemistry/domains
Loading domains from /nxfilter/tmp/BL/library/domains
Loading domains from /nxfilter/tmp/BL/webtv/domains
Loading domains from /nxfilter/tmp/BL/podcasts/domains
Loading domains from /nxfilter/tmp/BL/finance/insurance/domains
Loading domains from /nxfilter/tmp/BL/finance/trading/domains
Loading domains from /nxfilter/tmp/BL/finance/moneylending/domains
Loading domains from /nxfilter/tmp/BL/finance/banking/domains
Loading domains from /nxfilter/tmp/BL/finance/realestate/domains
Loading domains from /nxfilter/tmp/BL/finance/other/domains
Loading domains from /nxfilter/tmp/BL/alcohol/domains
Loading domains from /nxfilter/tmp/BL/searchengines/domains
Loading domains from /nxfilter/tmp/BL/dynamic/domains
Loading domains from /nxfilter/tmp/BL/politics/domains
Loading domains from /nxfilter/tmp/BL/automobile/planes/domains
Loading domains from /nxfilter/tmp/BL/automobile/bikes/domains
Loading domains from /nxfilter/tmp/BL/automobile/boats/domains
Loading domains from /nxfilter/tmp/BL/automobile/cars/domains
Loading domains from /nxfilter/tmp/BL/downloads/domains
Loading domains from /nxfilter/tmp/BL/military/domains
Loading domains from /nxfilter/tmp/BL/forum/domains
Loading domains from /nxfilter/tmp/BL/hacking/domains
Loading domains from /nxfilter/tmp/BL/webmail/domains
Loading domains from /nxfilter/tmp/BL/sex/lingerie/domains
Loading domains from /nxfilter/tmp/BL/sex/education/domains
Loading domains from /nxfilter/tmp/BL/porn/domains
Loading domains from /nxfilter/tmp/BL/hobby/gardening/domains
Loading domains from /nxfilter/tmp/BL/hobby/cooking/domains
Loading domains from /nxfilter/tmp/BL/hobby/games-misc/domains
Loading domains from /nxfilter/tmp/BL/hobby/games-online/domains
Loading domains from /nxfilter/tmp/BL/hobby/pets/domains
Loading domains from /nxfilter/tmp/BL/education/schools/domains
Loading domains from /nxfilter/tmp/BL/redirector/domains
Loading domains from /nxfilter/tmp/BL/recreation/martialarts/domains
Loading domains from /nxfilter/tmp/BL/recreation/wellness/domains
Loading domains from /nxfilter/tmp/BL/recreation/restaurants/domains
Loading domains from /nxfilter/tmp/BL/recreation/sports/domains
Loading domains from /nxfilter/tmp/BL/recreation/travel/domains
Loading domains from /nxfilter/tmp/BL/recreation/humor/domains
Loading domains from /nxfilter/tmp/BL/urlshortener/domains
Loading domains from /nxfilter/tmp/BL/homestyle/domains
Loading domains from /nxfilter/tmp/BL/adv/domains
Loading domains from /nxfilter/tmp/BL/fortunetelling/domains
Loading domains from /nxfilter/tmp/BL/models/domains
Loading domains from /nxfilter/tmp/BL/hospitals/domains
Loading domains from /nxfilter/tmp/BL/socialnet/domains
Loading domains from /nxfilter/tmp/BL/costtraps/domains
2000 (0.14%) inserted.
4000 (0.27%) inserted.
6000 (0.41%) inserted.
8000 (0.55%) inserted.
10000 (0.69%) inserted.
12000 (0.82%) inserted.
14000 (0.96%) inserted.
16000 (1.10%) inserted.
18000 (1.23%) inserted.
20000 (1.37%) inserted.
..... the last record is 100%

this is log nxfilter.log:
INFO [07-13 09:20:30] - Starting NxFilter v3.3.4
 INFO [07-13 09:20:30] - It's running as a master node.
 INFO [07-13 09:21:03] - Loading config.
 INFO [07-13 09:21:03] - Copying config into local DB.
 INFO [07-13 09:21:05] - Loading dns_setup.
 INFO [07-13 09:21:05] - ZoneDic.load_dynamic_domain, Loading dynamic domains.
 INFO [07-13 09:21:05] - Loading zone_file.
 INFO [07-13 09:21:05] - Loading alert.
 INFO [07-13 09:21:05] - Loading policy.
 INFO [07-13 09:21:05] - Loading grp.
 INFO [07-13 09:21:05] - Loading user.
 INFO [07-13 09:21:06] - Loading whitelist.
 INFO [07-13 09:21:06] - Loading wknown domains.
 INFO [07-13 09:21:06] - Loading whitelist_kw.
 INFO [07-13 09:21:06] - Loading category.
 INFO [07-13 09:21:06] - Loading category_domain.
 INFO [07-13 09:21:06] - NxClassifier.load_setup, Loading setup.
 INFO [07-13 09:21:06] - NxClassifier.load_rule_list, Loading ruleset.
 INFO [07-13 09:21:06] - ResolverMan.create_resolver, Resolving DNS server : [HIDE]
 INFO [07-13 09:21:06] - ResolverMan.create_resolver, Resolving DNS server : [HIDE]]
 INFO [07-13 09:21:06] - Loading allowed_ip.
 INFO [07-13 09:21:06] - Loading redirection.
 INFO [07-13 09:21:06] - Loading free_time.
 INFO [07-13 09:21:06] - Loading policy_application.
DEBUG [07-13 09:21:06] - PolicyApplication : -ec -ku -kt ei:60 e:[explorer.exe] e:[Program*Manager]
 INFO [07-13 09:21:06] - Loading policy_proxy.
DEBUG [07-13 09:21:06] - PolicyProxy : -ep fd:[HIDE]
 INFO [07-13 09:21:06] - Loading zone_file.
 INFO [07-13 09:21:06] - Copying config into local DB.
 INFO [07-13 09:21:06] - Copying block_page into local DB.
DEBUG [07-13 09:21:06] - Config{block_redi_ip=[HIDE], rf_block_redi_ip=[HIDE], login_domain=[HIDE], logout_domain=[HIDE], enable_login=true, log_retention_days=90, login_session_ttl=1440, clt_cache_ttl=0, syslog_host=[HIDE], export_blocked_only=false, remote_logging=false, use_netflow=false, netflow_ip=, netflow_port=2055, auto_backup_days=20, admin_domain=[HIDE], agent_policy_update_period=60, debug_flag=false, debug_level=0, rh_num=8, rq_size=1000, start_tomcat=true, http_port=80, https_port=443, dns_port=53, local_port=19001, login_port=19002, h2db_port=19003, node_port=19004, listen_ip=0.0.0.0, blacklist_type=1, category_table=category_shalla, domain_table=domain_shalla, adware_catid=1, log_blocked_only=false, partner_code=, demo_flag=false, upstream_dns_list=[], cluster_mode=1, master_ip=, master_flag=true, slave_flag=false, slave_ip=, slave_ip_arr=[], max_slave_num=4, www_dir=webapps, keystore_file=, keystore_pass=, report_server_ip=, report_server_port=19003, test_load_value=0, log_flush_limit=1000, most_permissive=false, syslog_only=false, stop_auto_report=false, ldap_conn_timeout=6, ldap_read_timeout=20, show_netflow=false, no_share_session=false, ipv6_ip=, free_time_flag=false, uid=JWTBWKC53656}
DEBUG [07-13 09:21:06] - DnsSetup{upstream_dns_arr=[10.8.3.10, 10.8.3.11, ], upstream_timeout=6, resp_cache_size=200000, clt_cache_ttl=0}
DEBUG [07-13 09:21:06] - ConfigAlert{admin_email=, smtp_host=, smtp_port=0, smtp_ssl=false, smtp_user=, smtp_passwd=, period=0}
 INFO [07-13 09:21:06] - ConfigLoader started.
 INFO [07-13 09:21:07] - LicenseChecker started.
 INFO [07-13 09:21:07] - BlockDomainResolver started.
DEBUG [07-13 09:21:07] - wk_dic.size() == 100004
DEBUG [07-13 09:21:06] - Config{block_redi_ip=[HIDE], rf_block_redi_ip=[HIDE], login_domain=[HIDE], logout_domain=[HIDE], enable_login=true, log_retention_days=90, login_session_ttl=1440, clt_cache_ttl=0, syslog_host=[HIDE], export_blocked_only=false, remote_logg
ing=false, use_netflow=false, netflow_ip=, netflow_port=2055, auto_backup_days=20, admin_domain=[HIDE], agent_policy_update_period=60, de
bug_flag=false, debug_level=0, rh_num=8, rq_size=1000, start_tomcat=true, http_port=80, https_port=443, dns_port=53, local_port=19001, login_port=19002, h2db_
port=19003, node_port=19004, listen_ip=0.0.0.0, blacklist_type=1, category_table=category_shalla, domain_table=domain_shalla, adware_catid=1, log_blocked_only
=false, partner_code=, demo_flag=false, upstream_dns_list=[], cluster_mode=1, master_ip=, master_flag=true, slave_flag=false, slave_ip=, slave_ip_arr=[], max_
slave_num=4, www_dir=webapps, keystore_file=, keystore_pass=, report_server_ip=, report_server_port=19003, test_load_value=0, log_flush_limit=1000, most_permi
ssive=false, syslog_only=false, stop_auto_report=false, ldap_conn_timeout=6, ldap_read_timeout=20, show_netflow=false, no_share_session=false, ipv6_ip=, free_
time_flag=false, uid=JWTBWKC53656}
DEBUG [07-13 09:21:06] - DnsSetup{upstream_dns_arr=[HIDE], upstream_timeout=6, resp_cache_size=200000, clt_cache_ttl=0}
DEBUG [07-13 09:21:06] - ConfigAlert{admin_email=, smtp_host=, smtp_port=0, smtp_ssl=false, smtp_user=, smtp_passwd=, period=0}
 INFO [07-13 09:21:06] - ConfigLoader started.
 INFO [07-13 09:21:07] - LicenseChecker started.
 INFO [07-13 09:21:07] - BlockDomainResolver started.
DEBUG [07-13 09:21:07] - wk_dic.size() == 100004
 INFO [07-13 09:21:07] - UserActDic initiated.
 INFO [07-13 09:21:07] - LogWriter started.
 INFO [07-13 09:21:07] - AlertMan started.
 INFO [07-13 09:21:07] - PostBox started.
 INFO [07-13 09:21:07] - LdapUpdater started.
 INFO [07-13 09:21:07] - LocalListener started.
DEBUG [07-13 09:21:07] - Zone transfer, [HIDE]
 INFO [07-13 09:21:07] - LoginListener started.
 INFO [07-13 09:21:07] - HandyMan started.
 INFO [07-13 09:21:07] - HandyMan.delete_old.
 INFO [07-13 09:21:07] - ReportMan started.
 INFO [07-13 09:21:07] - Reset user_quota.
 INFO [07-13 09:21:07] - Loading user_quota.
 INFO [07-13 09:21:07] - NodeListener started.
 INFO [07-13 09:21:07] - ZoneTransfer started.
 INFO [07-13 09:21:07] - DynUpdate.load_resolver.
DEBUG [07-13 09:21:07] - DynUpdate.load_resolver - DynData{domain=, arpa=.8.10.in-addr.arpa, host=[HIDE], dns_ip=[HIDE]1, dns_timeout=6, dns_load_balance=true}
 INFO [07-13 09:21:07] - RecatMan started.
 INFO [07-13 09:21:07] - ResolverMan.create_resolver, Resolving DNS server :[HIDE]
DEBUG [07-13 09:21:07] - Zone transfer, [HIDE]
 INFO [07-13 09:21:07] - RequestHandler started.
 INFO [07-13 09:21:07] - UdpServer started.
DEBUG [07-13 09:21:07] - Looking up for forcesafesearch.google.com
 INFO [07-13 09:21:07] - Starting TCP DB.
 INFO [07-13 09:21:07] - Starting Tomcat.
DEBUG [07-13 09:21:08] - RH #1, teredo.ipv6.microsoft.com, rq_size= 0, r_dc = 1, r_ttl = 0, r_type = 1

[Jinhee]

I don't see anything wrong. You see all the domains unclassified on your log-view and on 'Category > Domain Test' you get only 'unclassified' even if you test it with 'google.com'?

[Alessandro Peroni]

Hi Jinhee,
Yes i see unclassified also google.com and facebook.com or twitter.com or youporn.com.

If you want i can organize session with teamviewer or skype.

Thanks for your support
Best,
Alessandro

[Jinhee]

Show me the content of your /nxfilter/db then.

[Jinhee]

'ls /nxfilter/db' if you are on a Linux. And if you have 'blacklist.trace.db' show me the content of it.

[Alessandro Peroni]

i am using debian 8.5 stable.
root@...:/nxfilter/db# ls -alh
total 133M
drwxr-xr-x  2 root root 4.0K Jul 13 10:28 .
drwxr-xr-x 14 root root 4.0K Jul 12 21:54 ..
-rw-r--r--  1 root root 146M Jul 13 09:21 blacklist.h2.db
-rw-r--r--  1 root root  100 Jul 13 09:21 blacklist.lock.db
-rw-r--r--  1 root root  312 Jul 13 09:19 blacklist.trace.db
-rw-r--r--  1 root root  34K Jul 13 10:28 cache.h2.db
-rw-r--r--  1 root root  30K Jul 13 02:00 cache.trace.db
-rw-r--r--  1 root root 1.5M Jul 13 11:04 config.h2.db
-rw-r--r--  1 root root  100 Jul 13 09:20 config.lock.db
-rw-r--r--  1 root root 2.3K Jul 12 21:59 config.trace.db
-rw-r--r--  1 root root  29M Jul 13 09:21 jahaslist.h2.db
-rw-r--r--  1 root root  100 Jul 13 09:21 jahaslist.lock.db
-rw-r--r--  1 root root 1.1M Jul 13 09:21 local.h2.db
-rw-r--r--  1 root root  100 Jul 13 09:21 local.lock.db
-rw-r--r--  1 root root 5.9M Jul 13 14:24 traffic.h2.db
-rw-r--r--  1 root root  100 Jul 13 09:20 traffic.lock.db
-rw-r--r--  1 root root  17K Jul 13 10:28 traffic.trace.db


as you see i have blacklist.trace.db.
Thanks

[Jinhee]

Your Shallalist data is in blacklist.h2.db. 146M is perfectly normal. What's in blacklist.trace.db? It's a text file.

[Alessandro Peroni]

root@.....:/nxfilter/db# cat blacklist.trace.db
07-11 11:57:20 pageStore: Transaction log could not be truncated; size: 48 MB
07-12 18:58:52 pageStore: Transaction log could not be truncated; size: 48 MB
07-12 21:57:00 pageStore: Transaction log could not be truncated; size: 48 MB
07-13 09:19:57 pageStore: Transaction log could not be truncated; size: 48 MB
root@.....:/nxfilter/db# cat blacklist.lock.db
#FileLock
#Wed Jul 13 09:21:03 CEST 2016
id=155e321612cf03d6e70e8c94a5bdb66a66d14bf8285
method=file

[Jinhee]

Whatever it is you have some problem with the DB. Stop NxFilter and 'rm /nxfilter/db/blacklist.*' and then delete '/nxfilter/conf/shalla.md5' and then update Shallalist again then restart it.

[Alessandro Peroni]

I tried your solution but without success :-(.

root@.....:/nxfilter/db# ls -alh
total 135M
drwxr-xr-x  2 root root 4.0K Jul 13 14:51 .

drwxr-xr-x 14 root root 4.0K Jul 12 21:54 ..

-rw-r--r--  1 root root 110M Jul 13 14:51 blacklist.h2.db
-rw-r--r--  1 root root  100 Jul 13 14:51 blacklist.lock.db
-rw-r--r--  1 root root 1.1M Jul 13 14:51 cache.h2.db
-rw-r--r--  1 root root  100 Jul 13 14:51 cache.lock.db

-rw-r--r--  1 root root  30K Jul 13 02:00 cache.trace.db

-rw-r--r--  1 root root 1.5M Jul 13 14:52 config.h2.db
-rw-r--r--  1 root root  100 Jul 13 14:51 config.lock.db

-rw-r--r--  1 root root 2.3K Jul 12 21:59 config.trace.db

-rw-r--r--  1 root root  29M Jul 13 14:51 jahaslist.h2.db
-rw-r--r--  1 root root  100 Jul 13 14:51 jahaslist.lock.db
-rw-r--r--  1 root root 1.1M Jul 13 14:51 local.h2.db
-rw-r--r--  1 root root  100 Jul 13 14:51 local.lock.db
-rw-r--r--  1 root root 5.1M Jul 13 14:52 traffic.h2.db
-rw-r--r--  1 root root  100 Jul 13 14:51 traffic.lock.db

-rw-r--r--  1 root root  17K Jul 13 10:28 traffic.trace.db

Missing blacklist.trace.db.

nxfilter.log
DEBUG [07-13 14:50:12] - /A shutdown
 INFO [07-13 14:50:12] - shutting down system.
DEBUG [07-13 14:50:43] - old_md5 = , new_md5 = e7b6bf02d1d81db83aad958bddfe7dc5  shallalist.tar.gz
 INFO [07-13 14:50:43] - Downloading http://www.shallalist.de/Downloads/shallalist.tar.gz
 INFO [07-13 14:50:48] - Creating blacklist DB.
 INFO [07-13 14:51:34] - Starting NxFilter v3.3.4
 INFO [07-13 14:51:34] - It's running as a master node.
 INFO [07-13 14:51:48] - Loading config.
 INFO [07-13 14:51:48] - Copying config into local DB.
 INFO [07-13 14:51:50] - Loading dns_setup.
 INFO [07-13 14:51:50] - ZoneDic.load_dynamic_domain, Loading dynamic domains.
 INFO [07-13 14:51:50] - Loading zone_file.
 INFO [07-13 14:51:50] - Loading alert.
 INFO [07-13 14:51:50] - Loading policy.
 INFO [07-13 14:51:50] - Loading grp.
 INFO [07-13 14:51:50] - Loading user.
 INFO [07-13 14:51:50] - Loading whitelist.
 INFO [07-13 14:51:50] - Loading wknown domains.
 INFO [07-13 14:51:51] - Loading whitelist_kw.
 INFO [07-13 14:51:51] - Loading category.
 INFO [07-13 14:51:51] - Loading category_domain.
 INFO [07-13 14:51:51] - NxClassifier.load_setup, Loading setup.
 INFO [07-13 14:51:51] - NxClassifier.load_rule_list, Loading ruleset.
 INFO [07-13 14:51:51] - ResolverMan.create_resolver, Resolving DNS server : [[HIDE], [HIDE]]
 INFO [07-13 14:51:51] - ResolverMan.create_resolver, Resolving DNS server : [[HIDE], [HIDE]]
 INFO [07-13 14:51:51] - Loading allowed_ip.
 INFO [07-13 14:51:51] - Loading redirection.
 INFO [07-13 14:51:51] - Loading free_time.
 INFO [07-13 14:51:51] - Loading policy_application.
DEBUG [07-13 14:51:51] - PolicyApplication : -ec -ku -kt ei:60 e:[explorer.exe] e:[Program*Manager]
 INFO [07-13 14:51:51] - Loading policy_proxy.
DEBUG [07-13 14:51:51] - PolicyProxy : -ep fd:[hide]
 INFO [07-13 14:51:51] - Loading zone_file.
  INFO [07-13 14:51:51] - Loading zone_file.
 INFO [07-13 14:51:51] - Copying config into local DB.
 INFO [07-13 14:51:51] - Copying block_page into local DB.
DEBUG [07-13 14:51:51] - Config{block_redi_ip=[HIDE], rf_block_redi_ip=[HIDE], login_domain=[HIDE], logout_domain=[HIDE], enable_login=true, log_retention_days=90, login_session_ttl=1440, clt_cache_ttl=0, syslog_host=, export_blocked_only=false, remote_logging=false, use_netflow=false, netflow_ip=, netflow_port=2055, auto_backup_days=20, admin_domain=[HIDE], agent_policy_update_period=60, debug_flag=false, debug_level=0, rh_num=8, rq_size=1000, start_tomcat=true, http_port=80, https_port=443, dns_port=53, local_port=19001, login_port=19002, h2db_port=19003, node_port=19004, listen_ip=0.0.0.0, blacklist_type=1, category_table=category_shalla, domain_table=domain_shalla, adware_catid=1, log_blocked_only=false, partner_code=, demo_flag=false, upstream_dns_list=[], cluster_mode=1, master_ip=, master_flag=true, slave_flag=false, slave_ip=, slave_ip_arr=[], max_slave_num=4, www_dir=webapps, keystore_file=, keystore_pass=, report_server_ip=, report_server_port=19003, test_load_value=0, log_flush_limit=1000, most_permissive=false, syslog_only=false, stop_auto_report=false, ldap_conn_timeout=6, ldap_read_timeout=20, show_netflow=false, no_share_session=false, ipv6_ip=, free_time_flag=false, uid=JWTBWKC53656}
DEBUG [07-13 14:51:51] - DnsSetup{upstream_dns_arr=[[HIDE], [HIDE], ], upstream_timeout=6, resp_cache_size=200000, clt_cache_ttl=0}
DEBUG [07-13 14:51:51] - ConfigAlert{admin_email=, smtp_host=, smtp_port=0, smtp_ssl=false, smtp_user=, smtp_passwd=, period=0}
 INFO [07-13 14:51:51] - ConfigLoader started.
 INFO [07-13 14:51:51] - LicenseChecker started.
 INFO [07-13 14:51:51] - BlockDomainResolver started.
DEBUG [07-13 14:51:51] - wk_dic.size() == 100006
 INFO [07-13 14:51:51] - UserActDic initiated.
 INFO [07-13 14:51:51] - LogWriter started.
 INFO [07-13 14:51:51] - AlertMan started.
 INFO [07-13 14:51:51] - PostBox started.
 INFO [07-13 14:51:51] - LdapUpdater started.
 INFO [07-13 14:51:51] - LocalListener started.
DEBUG [07-13 14:51:51] - Zone transfer, [HIDE], [HIDE]
 INFO [07-13 14:51:52] - LoginListener started.
 INFO [07-13 14:51:52] - HandyMan started.
 INFO [07-13 14:51:52] - HandyMan.delete_old.
 INFO [07-13 14:51:52] - ReportMan started.
 INFO [07-13 14:51:52] - Reset user_quota.
 INFO [07-13 14:51:52] - Loading user_quota.
 INFO [07-13 14:51:52] - NodeListener started.
 INFO [07-13 14:51:52] - ZoneTransfer started.
 INFO [07-13 14:51:52] - DynUpdate.load_resolver.
DEBUG [07-13 14:51:52] - DynUpdate.load_resolver - DynData{domain=, arpa=.8.10.in-addr.arpa, host=[HIDE], dns_ip=[HIDE],[HIDE], dns_timeout=6, dns_load_balance=true}
 INFO [07-13 14:51:52] - RecatMan started.
 INFO [07-13 14:51:52] - ResolverMan.create_resolver, Resolving DNS server : [[HIDE], [HIDE]]
DEBUG [07-13 14:51:52] - Zone transfer, [HIDE], [HIDE]
 INFO [07-13 14:51:52] - RequestHandler started.
 INFO [07-13 14:51:52] - UdpServer started.
DEBUG [07-13 14:51:52] - Looking up for forcesafesearch.google.com
 INFO [07-13 14:51:52] - Starting TCP DB.
 INFO [07-13 14:51:52] - Starting Tomcat.

Have you another solution?

[Jinhee]

What if you switch to Jahaslist or how was it when you use Jahaslist? Did you get the classification? Then your problem happens only with blacklist.h2.db. Probably having a disk problem and making a problem with the DB? Do you have enough space for the DB? That 'trasacation' message means it was doing something but couldn't finish it. Might be indexing. And it requires some space.

[Alessandro Peroni]

I have same problem also with jahaslist.

I have not issue of space, it is VM on ESXi 5.5

root@..........:/nxfilter/db# df -h
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup01-ROOT                         14G  1.5G   12G  12% /
udev                              10M     0   10M   0% /dev
tmpfs                            1.6G   77M  1.5G   5% /run
tmpfs                            4.0G     0  4.0G   0% /dev/shm
tmpfs                            5.0M     0  5.0M   0% /run/lock
tmpfs                            4.0G     0  4.0G   0% /sys/fs/cgroup
/dev/sda1                        1.9G   34M  1.7G   2% /boot
/dev/mapper/VolGroup01-VAR_LOG   4.5G   56M  4.2G   2% /var/log
/dev/mapper/VolGroup01-NXFILTER   19G  251M   18G   2% /nxfilter
tmpfs                            801M     0  801M   0% /run/user/1610201835

[Alessandro Peroni]

I resolved!!! I had issue with my configuration. What i done?

1. Clean all folder nxfilter
2. reinstall package .deb
3. update BL Shallalist
4. Backup config.h2.db original
5. Restore my backup
6. There was still the BL issue's
7. Restore config.h2.db original
8. Blacklist it's OK!
9. Reconfigure all setting on nxfilter
10. Test OK
11. Enjoy :-D

Thanks for your support Jinhee and carry on this way!

Best,
Alessandro

[Claes Boström]

Hi. I have encountered the same issue as the thread creator. For some reason NxFilter started considering everything in my Shallalist "unclassified" a couple of days ago, seemingly by itself - I had not touched the server for a long time. I did have an unexpected crash a while ago (power outage), perhaps that may have something to do with it. Anyway, I have tried to rebuild the blacklist (as described in this thread), I have reinstalled NxFilter etc but I still get the same result - any dns record that should have been classified is not, but all records I have manually added are caught as expected. I was on version 3.0.4 when I discovered the issue a few days ago, I am now on 3.4.0.

The only way I can get it to work is (like in the previous post) to install NxFilter completely from scratch and use the default configuration. If I replace the default config with my old config.h2.db the behaviour with "unclassified" comes back. So now I have two versions of config.h2.db - one works and one doesn't on the same installation of NxFilter.

It is quite painful to do all the setup again. Since I am not the only one seeing this issue (well, there is at least one more :) perhaps I can email you the config files and you can find out what the issue may be? It must be something I can't see through the GUI...

Apart from this issue the system has worked like a charm for a couple of years!

/Claes

[Jinhee]

OK. Send me your config DB. 'support at nxfilter.org'.

[Jinhee]

It's because on one of your AD setup you have a blank domain and all the domains being recognized as a local domain. We bypass classification for a local domain. So add some domain there.