Re: [security-onion] Adding Parses to ELSA

[Martin Holste]

The ELSA mailing list is a Google Group.  There's a link from the ELSA project page to it, and I encourage you to join.

The field_order is the column number for the syslogs tables if you started counting with id as zero, timestamp as one, all the way until i0-i5 are 5-10, and s0-s5 are 11-16.  Take a look at the syslog.syslogs_template table (desc syslogs_template) to see what the raw table itself looks like.

You only need to add fields to the fields table if the name doesn't exist already.  Each field there will have an id, and that id is used as a reference in the fields_classes_map table, which ties the field to the class and holds the field_order which was described above.

Hopefully that helps, let me know if you still have questions, and check out the ELSA mailing list for previous examples and assistance I've given.

On Sun, Jun 9, 2013 at 4:24 PM, <offe...@gmail.com> wrote:

Hi All,

I don't know if this is the place to ask but i don't see any ELSA forum?? please point me in the right direction if i missed it.

Im trying to create a new parser og class if you will.

I have read the documentation at the ELSA site and i think i understand most of it
https://code.google.com/p/enterprise-log-search-and-archive/wiki/Documentation#Adding_Parsers

Now the part that i have trouble understanding is:

Fields creation
and
the use of i0 i1 i2 s0

First Field creation:

quote from documentation:

Our fields will be conn_bytes, srcip, and dstip, which already exist in the "fields" table as well as "myuser" which we will create here for demonstration purposes:

INSERT INTO fields (field, field_type, pattern_type) VALUES ("myuser",
"string", "QSTRING");

INSERT INTO fields_classes_map (class_id, field_id, field_order)
VALUES ((SELECT id FROM classes WHERE class="NEWCLASS"), (SELECT
id FROM fields WHERE field="srcip"), 5);
INSERT INTO fields_classes_map (class_id, field_id, field_order)
VALUES ((SELECT id FROM classes WHERE class="NEWCLASS"), (SELECT
id FROM fields WHERE field="conn_bytes"), 6);
INSERT INTO fields_classes_map (class_id, field_id, field_order)
VALUES ((SELECT id FROM classes WHERE class="NEWCLASS"), (SELECT
id FROM fields WHERE field="dstip"), 7);
Now the string field "myuser" at field_order 11, which maps to the first string column "s0":

INSERT INTO fields_classes_map (class_id, field_id, field_order)
VALUES ((SELECT id FROM classes WHERE class="NEWCLASS"), (SELECT
id FROM fields WHERE field="myuser"), 11);

Quote end

We have chosen to use source ip, destination ip and byte count which already exist in the database with the field_order og 5 6 and 7. How do we know that? Can i look it up somehow? I may need to make a class with another field that may  allready exist and so i need to know its filed_order number.

Also we choose to create a new filed called myuser and give it field_order number of 11. The documentation state we do this because it would match the first string column  of s0.

What is s0? and how should use this in the future if I have more custom fields i need to create?

also field_number 5 6 7 is said to match i0 i1 i2 in the schema. How should I relate to this if I have more predefined fields then 3?

I really love ELSA and what it gives me and learning to do this brings it to a whole new level!!

A big thank your to the ELSA team out there... outstanding work!

Thanks

Casper

--
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at http://groups.google.com/group/security-onion?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.

[Martin Holste]

So to inspect the syslog_template table, here are the exact steps:

mysql -uroot -p syslog

show create table syslogs_template;

exit

To find out an id for a field:

mysql -uroot -p syslog

select id from fields where field="somefield";

To find out the id of a class:

mysql -uroot -p syslog

select id from classes where class="someclass"

Make sure the class is all upper-case.

To create a new class:

mysql -uroot -p syslog

select * from classes where id > 1000;

That will print any custom classes.  Classes you create should start over 1000.  So if you haven't created any, you can use 1000 as the first id.

insert into classes (id, class) values (1000, "MYCLASS");

This would insert a field called "somefield" for class "MYCLASS" as the first integer field, i0:

insert into fields_classes_map (field_id, class_id, field_order) values ((select id from fields where field="somefield"), (select id from classes where class="MYCLASS"), 5);

You would repeat that for each field in the class.

And yes, someday I will have a web interface for this.

On Mon, Jun 10, 2013 at 12:39 PM, <offe...@gmail.com> wrote:

Arg... im so not used to this!!

ok so i played around the mysql server of SO and god it took me a while to figure out i had to end my statements with a ; before i would accept them. When i got through that i tried a show databases; that got me a database test and information_schema where the only one showing any tables was information_schema but none of what you mention Martin?

this is the output of information_schema:

+---------------------------------------+
| Tables_in_information_schema          |
+---------------------------------------+
| CHARACTER_SETS                        |
| COLLATIONS                            |
| COLLATION_CHARACTER_SET_APPLICABILITY |
| COLUMNS                               |
| COLUMN_PRIVILEGES                     |
| ENGINES                               |
| EVENTS                                |
| FILES                                 |
| GLOBAL_STATUS                         |
| GLOBAL_VARIABLES                      |
| KEY_COLUMN_USAGE                      |
| PARAMETERS                            |
| PARTITIONS                            |
| PLUGINS                               |
| PROCESSLIST                           |
| PROFILING                             |
| REFERENTIAL_CONSTRAINTS               |
| ROUTINES                              |
| SCHEMATA                              |
| SCHEMA_PRIVILEGES                     |
| SESSION_STATUS                        |
| SESSION_VARIABLES                     |
| STATISTICS                            |
| TABLES                                |
| TABLESPACES                           |
| TABLE_CONSTRAINTS                     |
| TABLE_PRIVILEGES                      |
| TRIGGERS                              |
| USER_PRIVILEGES                       |
| VIEWS                                 |
| INNODB_BUFFER_PAGE                    |
| INNODB_TRX                            |
| INNODB_BUFFER_POOL_STATS              |
| INNODB_LOCK_WAITS                     |
| INNODB_CMPMEM                         |
| INNODB_CMP                            |
| INNODB_LOCKS                          |
| INNODB_CMPMEM_RESET                   |
| INNODB_CMP_RESET                      |
| INNODB_BUFFER_PAGE_LRU                |
+---------------------------------------+

what am i doing wrong??

[Doug Burks]

https://code.google.com/p/security-onion/wiki/Passwords#MySQL
Doug

On Wed, Jun 12, 2013 at 12:57 PM, <offe...@gmail.com> wrote:
> Hi Martin,
>
> my problem start with the command: mysql -uroot -p syslog
>
> now when i installed SO i choose support as my username. So i went with -usupport and tried both the password i use to log into the shell and the password for SO none of them worked..
>
> Is there some hidden user password in SO i don't know about or?
>
> /Casper

>
> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion?hl=en-US.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
Doug Burks
http://securityonion.blogspot.com

[offe...@gmail.com]

Thanks you two!!!

I think i got what i need to learn the rest on my own..

THANKS!...again

/Casper