File extraction truncated

66 views
Skip to first unread message

Scap

unread,
Jun 28, 2019, 10:35:02 AM6/28/19
to SELKS
Hello,

I just install the last version of SELKS OVF (8vCPU - 12 Go RAM, vmxnet3 VMware esxi) with suricata 5.0.0-beta1 (i also test with the stable 4 version)

I activate the extraction

I have some rules to trig .exe files or html, css, etc.... But i only get TRUNCATED for all .exe. Only the little file like a web page html is extract.

Any idea?

Kind regards

Scap

Peter Manev

unread,
Jun 28, 2019, 10:47:47 AM6/28/19
to Scap, SELKS
On Fri, Jun 28, 2019 at 5:35 PM Scap <scar...@gmail.com> wrote:
>
> Hello,
>
> I just install the last version of SELKS OVF (8vCPU - 12 Go RAM, vmxnet3 VMware esxi) with suricata 5.0.0-beta1 (i also test with the stable 4 version)
>

SELKS comes with higher version then the 5.0.0-beta - so i suppose you
have done your own install of Suri, right? (it's no problem - just
checking)

> I activate the extraction
>
> I have some rules to trig .exe files or html, css, etc.... But i only get TRUNCATED for all .exe. Only the little file like a web page html is extract.
>
> Any idea?

Depending on the file size you should adjust the following settings -
https://suricata.readthedocs.io/en/latest/file-extraction/file-extraction.html?store#settings

Thank you

>
> Kind regards
>
> Scap
>
> --
> IRC: Let's talk about SELKS on Freenode IRC on the #SELKS channel
> Wiki: https://github.com/StamusNetworks/SELKS/wiki
> GitHub: https://github.com/StamusNetworks/SELKS
> Blog: https://www.stamus-networks.com/theblog/
> Twitter: @StamusN
> g+: Stamus Networks
> ---
> You received this message because you are subscribed to the Google Groups "SELKS" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to selks+un...@googlegroups.com.
> To post to this group, send email to se...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/selks/647d19f6-d00c-4981-89cf-aa93ad68d454%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Regards,
Peter Manev

Scap

unread,
Jun 28, 2019, 10:54:50 AM6/28/19
to SELKS
Hello!

I didn't check, i get the problem before build the version 4 in local and after i download the 5 from the website too.

So i got the same result for the 3 versions

Kind regards

Le vendredi 28 juin 2019 16:47:47 UTC+2, pevma a écrit :
On Fri, Jun 28, 2019 at 5:35 PM Scap <scar...@gmail.com> wrote:
>
> Hello,
>
> I just install the last version of SELKS OVF (8vCPU - 12 Go RAM, vmxnet3 VMware esxi) with suricata 5.0.0-beta1 (i also test with the stable 4 version)
>

SELKS comes with higher version then the 5.0.0-beta - so i suppose you
have done your own install of Suri, right? (it's no problem - just
checking)

> I activate the extraction
>
> I have some rules to trig .exe files or html, css, etc.... But i only get TRUNCATED for all .exe. Only the little file like a web page html is extract.
>
> Any idea?

Depending on the file size you should adjust the following settings -
https://suricata.readthedocs.io/en/latest/file-extraction/file-extraction.html?store#settings

Thank you

>
> Kind regards
>
> Scap
>
> --
> IRC: Let's talk about SELKS on Freenode IRC on the #SELKS channel
> Wiki: https://github.com/StamusNetworks/SELKS/wiki
> GitHub: https://github.com/StamusNetworks/SELKS
> Blog: https://www.stamus-networks.com/theblog/
> Twitter: @StamusN
> g+: Stamus Networks
> ---
> You received this message because you are subscribed to the Google Groups "SELKS" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to se...@googlegroups.com.

Peter Manev

unread,
Jun 28, 2019, 11:05:33 AM6/28/19
to Scap, SELKS
On Fri, Jun 28, 2019 at 5:54 PM Scap <scar...@gmail.com> wrote:
>
> Hello!
>
> I didn't check, i get the problem before build the version 4 in local and after i download the 5 from the website too.
>
> So i got the same result for the 3 versions
>

Ok - please have a look at the docs and see if it helps.
what files are you trying to extract - http/ftp/smb based ?
> To unsubscribe from this group and stop receiving emails from it, send an email to selks+un...@googlegroups.com.
> To post to this group, send email to se...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/selks/d88a374a-9590-4c4b-a7af-a164e727e9cf%40googlegroups.com.

Damien HOTZ

unread,
Jun 28, 2019, 3:26:45 PM6/28/19
to Peter Manev, Scap, SELKS
Only .exe download by http

To view this discussion on the web visit https://groups.google.com/d/msgid/selks/CAMhe82KyzckDEyjSjM%2BWvgDJCjLJDRSrehiGQbD9cMMNJC-UwA%40mail.gmail.com.

Scap

unread,
Jul 1, 2019, 3:35:38 AM7/1/19
to SELKS
As i was thinking, i just change the HTTP body size and now i have extracted my .exe file.

I am going to test other file, but i guess it was the problem :)

Sorry for this usuless question, i was on the suricata customisation page but i didn't see the HTTP body fields.

Kind regards

Scap
> To view this discussion on the web visit https://groups.google.com/d/msgid/selks/d88a374a-9590-4c4b-a7af-a164e727e9cf%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Regards,
Peter Manev

--
IRC: Let's talk about SELKS on Freenode IRC on the #SELKS channel
Wiki: https://github.com/StamusNetworks/SELKS/wiki
GitHub: https://github.com/StamusNetworks/SELKS
Blog: https://www.stamus-networks.com/theblog/
Twitter: @StamusN
g+: Stamus Networks
---
You received this message because you are subscribed to the Google Groups "SELKS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to se...@googlegroups.com.
To post to this group, send email to se...@googlegroups.com.

Peter Manev

unread,
Jul 2, 2019, 9:20:16 AM7/2/19
to Scap, SELKS
On Mon, Jul 1, 2019 at 10:35 AM Scap <scar...@gmail.com> wrote:
>
> As i was thinking, i just change the HTTP body size and now i have extracted my .exe file.
>
> I am going to test other file, but i guess it was the problem :)
>

Yes, those would need to be adjusted.
> To unsubscribe from this group and stop receiving emails from it, send an email to selks+un...@googlegroups.com.
> To post to this group, send email to se...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/selks/4ac920dc-57f3-4051-b10e-8887b57d4852%40googlegroups.com.

Scap

unread,
Jul 16, 2019, 8:19:39 AM7/16/19
to SELKS
Hi,

I got have truncated file now.

I can extract file like 24Ko or 512Ko

But i cannot with 4Mo or 8Mo

this is my configuration :


And an example :


Peter Manev

unread,
Jul 16, 2019, 8:27:05 AM7/16/19
to Scap, SELKS
On Tue, Jul 16, 2019 at 2:19 PM Scap <scar...@gmail.com> wrote:
Hi,

I got have truncated file now.

I can extract file like 24Ko or 512Ko

But i cannot with 4Mo or 8Mo

you probably need to adjust - https://github.com/OISF/suricata/blob/master/suricata.yaml.in#L1347 as well.
Also check the filestore settings that are set to the appropriate number - https://github.com/OISF/suricata/blob/master/suricata.yaml.in#L471

I think those can be adjusted in the selks5-addin.yaml, save and exit then restart suri

Thank you
 

this is my configuration :


And an example :


--
IRC: Let's talk about SELKS on Freenode IRC on the #SELKS channel
Wiki: https://github.com/StamusNetworks/SELKS/wiki
GitHub: https://github.com/StamusNetworks/SELKS
Blog: https://www.stamus-networks.com/theblog/
Twitter: @StamusN
g+: Stamus Networks
---
You received this message because you are subscribed to the Google Groups "SELKS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selks+un...@googlegroups.com.
To post to this group, send email to se...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--
Regards,
Peter Manev

Scap

unread,
Jul 16, 2019, 8:46:58 AM7/16/19
to SELKS
Thank you for this quick answer.

I don't have the depth option in selks yaml but i have modify it in suricata.yaml
so i uncomment the # depth line.

and i am already at 0 for the second option.

but it is not working


Le mardi 16 juillet 2019 14:27:05 UTC+2, pevma a écrit :
On Tue, Jul 16, 2019 at 2:19 PM Scap <scar...@gmail.com> wrote:
Hi,

I got have truncated file now.

I can extract file like 24Ko or 512Ko

But i cannot with 4Mo or 8Mo

you probably need to adjust - https://github.com/OISF/suricata/blob/master/suricata.yaml.in#L1347 as well.
Also check the filestore settings that are set to the appropriate number - https://github.com/OISF/suricata/blob/master/suricata.yaml.in#L471

I think those can be adjusted in the selks5-addin.yaml, save and exit then restart suri

Thank you
 

this is my configuration :


And an example :


--
IRC: Let's talk about SELKS on Freenode IRC on the #SELKS channel
Wiki: https://github.com/StamusNetworks/SELKS/wiki
GitHub: https://github.com/StamusNetworks/SELKS
Blog: https://www.stamus-networks.com/theblog/
Twitter: @StamusN
g+: Stamus Networks
---
You received this message because you are subscribed to the Google Groups "SELKS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to se...@googlegroups.com.

To post to this group, send email to se...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selks/b26ad53e-c155-4d26-87c9-04bc4dc391be%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Regards,
Peter Manev

Peter Manev

unread,
Jul 16, 2019, 8:49:33 AM7/16/19
to Scap, SELKS
On Tue, Jul 16, 2019 at 2:47 PM Scap <scar...@gmail.com> wrote:
Thank you for this quick answer.

I don't have the depth option in selks yaml but i have modify it in suricata.yaml
so i uncomment the # depth line.

and i am already at 0 for the second option.

but it is not working



probably yaml grammar mistake.
what is on line 1317 ?

Thank you
 
To unsubscribe from this group and stop receiving emails from it, send an email to selks+un...@googlegroups.com.

To post to this group, send email to se...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--
Regards,
Peter Manev

Scap

unread,
Jul 16, 2019, 8:53:02 AM7/16/19
to SELKS
yeh, but if i uncomment the line fot the depth it is not started. As you can see the error previously.



--
Regards,
Peter Manev

Scap

unread,
Jul 16, 2019, 9:03:46 AM7/16/19
to SELKS
Ok, i find it, it was just after all this section ... line 1355

Scap

unread,
Jul 16, 2019, 9:47:14 AM7/16/19
to SELKS
With this, i manage to extract less than 5mb (not all the time) but for 8,5mb it's really not working.

Any idea?

Peter Manev

unread,
Jul 16, 2019, 10:03:19 AM7/16/19
to Scap, SELKS
On Tue, Jul 16, 2019 at 3:47 PM Scap <scar...@gmail.com> wrote:
With this, i manage to extract less than 5mb (not all the time) but for 8,5mb it's really not working.

Any idea?


These are very high settings for dept that can impact your performance too on live traffic as well.   
What you could try to do first is to do tcpdump on the interface and open the pcap with wireshark - see if you do not have out of order/wrong chksum packets etc  
 
To unsubscribe from this group and stop receiving emails from it, send an email to selks+un...@googlegroups.com.

To post to this group, send email to se...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--
Regards,
Peter Manev

Scap

unread,
Jul 16, 2019, 10:44:12 AM7/16/19
to SELKS
I check the scirius interface on the Suricata dashboard and i get kernel drop ...


--
Regards,
Peter Manev

Scap

unread,
Jul 18, 2019, 8:41:05 AM7/18/19
to SELKS
I was in pool ressources, so i guess the cpu limitation for burst didn't help.

I am out the pool and it's  working better :)

Peter Manev

unread,
Jul 18, 2019, 8:54:26 AM7/18/19
to Scap, SELKS


On 18 Jul 2019, at 13:41, Scap <scar...@gmail.com> wrote:

I was in pool ressources, so i guess the cpu limitation for burst didn't help.

I am out the pool and it's  working better :)


What does pool mean ? (Sorry not understanding )


To unsubscribe from this group and stop receiving emails from it, send an email to selks+un...@googlegroups.com.

To post to this group, send email to se...@googlegroups.com.

Scarpafo Scarpafo

unread,
Jul 18, 2019, 8:55:24 AM7/18/19
to Peter Manev, SELKS
Sorry.
Pool ressources on VMware :)

Peter Manev

unread,
Jul 18, 2019, 9:10:49 AM7/18/19
to Scarpafo Scarpafo, SELKS

On 18 Jul 2019, at 13:55, Scarpafo Scarpafo <scar...@gmail.com> wrote:

Sorry.
Pool ressources on VMware :)

Aha ok
Thanks 
Reply all
Reply to author
Forward
0 new messages