Trying to understand CDR Client

Te Matau

unread,

Oct 9, 2015, 12:39:13 AM10/9/15

to CGRateS

Apologies in advance for silly questions. I'm fumbling around in the dark...

1. I'm trying to understand the content_fields within the cdrc section of the config files.

Is there a description of what each available cdr_field_id means and what its usage is? For example what is the subject used for? In the docs it refers "auto-detection of duplicated CDRs at server side based on accid and host fields". But the host field is not listed in the example.
In the example what does "value": "^*voice"do? Will that take anything in the cdr and append "voice" to it?

2. My CDRs have a lot more fields than listed in the example config. According to the docs the format of extra fields is <label_extrafield_1>:<index_extrafield_1> but that doesn't make much sense to me. Assuming we have a cdr format as follows: id, call_id, caller_id_number. Would content_fields look like:

{"tag": "extra1"},

{"tag": "accid", "cdr_field_id": "accid", "type": "cdrfield", "value": "10", "mandatory": true},

{"tag": "extra2"}

3. Is there a way to "normalise" call records? For example a user could dial 1234567 or 8581234567 or 18581234567. All should be normalised to +18581234567 and rated the same. Is there a way to do that or does the normalisation have to be done elsewhere?

4. Is the direction field in the RatingProfiles just based on a match or is it a fixed set of values? For example if the cdr has "outbound" should Direction in RatingProfiles be outbound?

5. Is there some up to date documentation of the destinationrates table? This is out of date based on the example in the tutorial.

6. Is there a way to skip the first line in the csv file which is a header? This was asked a while ago but I can't see it documented anywhere.

Thanks in advance for any assistance.

Iwada Eja

unread,

Oct 9, 2015, 2:13:54 AM10/9/15

to cgr...@googlegroups.com

Hi Matua, i'll try to answer the much i can while we wait for DanB or rif

3. Is there a way to "normalise" call records? ......

If using Freeswitch, for this you can use mod_translate to doing the necessary translations or 'normalising' as you put it. Another Option might be to put all those numbers as matching destinations,Obviously using Mod_translate would be better here.

4. Is the direction field in the RatingProfiles just based on a match or is it a fixed set of values.....

I think it's a preset set of Values, *out for outbound calls, and *in for inbound calls.

5. Is there some up to date documentation of the destinationrates table?.....

The DestinationsRates CSV file at the link is not up to date. CGRateS is evolving real fast and most ducumentation lags behind. I'ld step in and see if i can do some updates to the Documentation. In the mean while, you can install CGRateS and check

/usr/share/cgrates/tariffplans/tutorial/ for a more current CSV files. It might seem involved at first, but once you play around with the CSV files, you get a better understanding of what each column does.

6. Is there a way to skip the first line in the csv file which is a header?....

Mind throwing more light on why you might want that skipped?

You can always check out IRC #cgrates for a more interactive session

--
You received this message because you are subscribed to the Google Groups "CGRateS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cgrates+u...@googlegroups.com.
To post to this group, send email to cgr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cgrates/bd3876d4-2a86-47d0-90e1-351915aa9f59%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kind Regards
Iwada

DanB

unread,

Oct 9, 2015, 3:58:48 AM10/9/15

to cgr...@googlegroups.com

Hi Te,

Answers inline ...

On 09.10.2015 06:39, Te Matau wrote:

Apologies in advance for silly questions. I'm fumbling around in the dark...

1. I'm trying to understand the content_fields within the cdrc section of the config files.

Is there a description of what each available cdr_field_id means and what its usage is? For example what is the subject used for? In the docs it refers "auto-detection of duplicated CDRs at server side based on accid and host fields". But the host field is not listed in the example.

The cdr_field_id will map from your own CDR fields (.csv of .fwv) to CGRateS internal CDR field. If the field is not found in primary fields (eg: mysql table cdrs_primary) it will be automatically added as extra field. You can find the list of primary fields which you can use here:
https://github.com/cgrates/cgrates/blob/master/cdrc/cdrc.go#L46
and their string translation here:
https://github.com/cgrates/cgrates/blob/master/utils/consts.go

They will map to fields inside the internal CDR object here (you can also find field description inthere):
https://github.com/cgrates/cgrates/blob/master/engine/storedcdr.go#L68

In the example what does "value": "^*voice"do? Will that take anything in the cdr and append "voice" to it?

Will set the TOR to *voice. The format of defining the templates is described here (RSRField):
http://cgrates.readthedocs.org/en/latest/cdrclient.html#import-templates

2. My CDRs have a lot more fields than listed in the example config. According to the docs the format of extra fields is <label_extrafield_1>:<index_extrafield_1> but that doesn't make much sense to me. Assuming we have a cdr format as follows: id, call_id, caller_id_number. Would content_fields look like:

{"tag": "extra1"},

{"tag": "accid", "cdr_field_id": "accid", "type": "cdrfield", "value": "10", "mandatory": true},

{"tag": "extra2"}

No problem if you need to capture extra to what we already defined. As said they will end up in CdrsExtra.

3. Is there a way to "normalise" call records? For example a user could dial 1234567 or 8581234567 or 18581234567. All should be normalised to +18581234567 and rated the same. Is there a way to do that or does the normalisation have to be done elsewhere?

Yes, via the RSR templates. Check the above link.

4. Is the direction field in the RatingProfiles just based on a match or is it a fixed set of values? For example if the cdr has "outbound" should Direction in RatingProfiles be outbound?

For the moment we only support "^*out" as direction but planning already to improve that in the near future.

5. Is there some up to date documentation of the destinationrates table? This is out of date based on the example in the tutorial.

I would go with reading the tag names in the tutorial for now. Should be pretty straight forward. Will try to improve that in the near future. Please open an issue on github so we don't forget about it.

6. Is there a way to skip the first line in the csv file which is a header? This was asked a while ago but I can't see it documented anywhere.

Should be if you add comment character (#).

Thanks in advance for any assistance.

Sure,
DanB

Te Matau

unread,

Oct 9, 2015, 12:59:51 PM10/9/15

to CGRateS

Thanks, just one follow up at the moment:

"we only support "^*out" as direction"

In an earlier post I asked about rating inbound calls. The reply referred to Category which I think is in RatesProfile. The direction field in the CDR will be either inbound" or "outbound". But if the only possible Direction is *out, then I don't understand how it is possible to rate inbound calls?

DanB

unread,

Oct 9, 2015, 1:06:54 PM10/9/15

to cgr...@googlegroups.com

Hi Te,

For now the way to bill inbound is reverting subject and destination in your CDR. Category is optional to make sure the prices do not overlap with standard costs -eg: User A calling user B on outbound and both have attached costs, when user A will receive inbound and he needs to pay for it, you still need to make it look like user A is the subject and user B is the destination, although user B is the one dialing user A. Without the category you cannot split the costs for the case when user A is really dialing out and when he receives inbound calls.

Hope it makes sense.

DanB

--
You received this message because you are subscribed to the Google Groups "CGRateS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cgrates+u...@googlegroups.com.
To post to this group, send email to cgr...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cgrates/43e2a24f-3fb8-4c43-8551-91bb62b84ab5%40googlegroups.com.

Te Matau

unread,

Oct 14, 2015, 3:45:11 AM10/14/15

to CGRateS

I've made some progress and have cgr-engine starting and processing csv files:

Oct 14 07:28:56 vultr CGRateS[6068]: <Cdrc> Parsing: /etc/cgrates/test/in/cdr_eg_01.csv

Oct 14 07:28:56 vultr CGRateS[6068]: Finished processing cdr_eg_01.csv, moved to /etc/cgrates/test/out/cdr_eg_01.csv. Total records processed: 9, CDRs posted: 9, run duration: 29.852646ms

But it's not populating the data correctly so I think my config is wrong:

...

{"tag": "tor", "cdr_field_id": "TOR", "type": "cdrfield", "value": "^*voice", "mandatory": true},

{"tag": "accid", "cdr_field_id": "AccId", "type": "cdrfield", "value": "1", "mandatory": true},

{"tag": "reqtype", "cdr_field_id": "ReqType", "type": "cdrfield", "value": "^*postpaid", "mandatory": true},

{"tag": "direction", "cdr_field_id": "Direction", "type": "cdrfield", "value": "^*out", "mandatory": true},

{"tag": "tenant", "cdr_field_id": "Tenant", "type": "cdrfield", "value": "^tenant.co", "mandatory": true},

{"tag": "category", "cdr_field_id": "Category", "type": "cdrfield", "value": "^call", "mandatory": true},

{"tag": "account", "cdr_field_id": "Account", "type": "cdrfield", "value": "^b20cc94c158666e83c546c0d75103f19", "mandatory": true},

{"tag": "subject", "cdr_field_id": "Subject", "type": "cdrfield", "value": "19", "mandatory": true},

{"tag": "destination", "cdr_field_id": "Destination", "type": "cdrfield", "value": "18", "mandatory": true},

{"tag": "setup_time", "cdr_field_id": "SetupTime", "type": "cdrfield", "value": "21", "mandatory": true},

{"tag": "usage", "cdr_field_id": "Usage", "type": "cdrfield", "value": "7", "mandatory": true},

...

Is there anything obviously wrong with this?

DanB

unread,

Oct 14, 2015, 5:14:24 AM10/14/15

to cgr...@googlegroups.com

Hi Te,

Your template looks about right. Would need a sample of cdr and output you get vs what you expect, in order to tell you more.

DanB

--
You received this message because you are subscribed to the Google Groups "CGRateS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cgrates+u...@googlegroups.com.
To post to this group, send email to cgr...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cgrates/26a907d4-616b-422d-b1a9-78a3058c59b5%40googlegroups.com.

Te Matau

unread,

Oct 14, 2015, 1:46:39 PM10/14/15

to CGRateS

The input CDR is:

#id,call_id,caller_id_number,caller_id_name,callee_id_number,callee_id_name,duration_seconds,billing_seconds,timestamp,hangup_cause,other_leg_call_id,owner_id,to,from,direction,request,authorizing_id,cost,dialed_number,calling_from,datetime,unix_timestamp,rfc_1036,iso_8601,call_type,rate,rate_name,bridge_id,recording_url,call_priority,billing_party,other_party,date,time,duration,price

201510-36f118b4-6e29-11e5-ab0a-05aefe5670df,36f118b4-6e29-11e5-ab0a-05aefe5670df,18582224530,User 01,18581246308,18581246308,100,89,63611575199,NORMAL_CLEARING,15230...@192.168.1.63,,+18581...@222.222.222.222,user_...@1000114.tenant.co,outbound,+18581...@222.222.222.222,,0,18581246308,18582224530,9/10/2015 1:59:00,1444355999,"Fri, 09 Oct 2015 01:59:59 GMT",9/10/2015,,0,,15230...@192.168.1.63,,,1800111111,18581246308,9/10/2015,1:58:30,89,0.0386

201510-1...@192.168.1.63,15230...@192.168.1.63,user_pnoe9s,User 01,18581246308,18581246308,101,89,63611575199,NORMAL_CLEARING,36f118b4-6e29-11e5-ab0a-05aefe5670df,3f95e66169ef2ad8018946e81e226a00,12129...@1000114.tenant.co,user_...@1000114.tenant.co,inbound,12129...@1000114.tenant.co,ba869d4ed91719e5c8cc35a4536e964e,0,12129749744,user_pnoe9s,9/10/2015 1:59:00,1444355999,"Fri, 09 Oct 2015 01:59:59 GMT",9/10/2015,,0,,15230...@192.168.1.63,,,,,,,,

201510-c29973c2-6e27-11e5-a354-8bf952d26c75,c29973c2-6e27-11e5-a354-8bf952d26c75,18589779977,Reception,18589779977,Reception,534,525,63611575009,NORMAL_CLEARING,4c22ba361214f54a...@111.111.111.111,87137f5f3606e5312922467072c41924,user_...@1000114.tenant.co,18589...@111.111.111.111,outbound,user_...@1000114.tenant.co,1174f35055df6cc9426a3c5d3bab403e,0,user_8wvps5,18589779977,9/10/2015 1:56:00,1444355809,"Fri, 09 Oct 2015 01:56:49 GMT",9/10/2015,,0,,,,,,,,,,

201510-4c22ba361214...@111.111.111.111,4c22ba361214f54a...@111.111.111.111,18589779977,Reception,18589779977,Reception,548,547,63611575009,NORMAL_CLEARING,c29973c2-6e27-11e5-a354-8bf952d26c75,,18001...@222.222.222.222:5060,18589...@111.111.111.111,inbound,18001...@222.222.222.222,,0,1800111111,18589779977,9/10/2015 1:56:00,1444355809,"Fri, 09 Oct 2015 01:56:49 GMT",9/10/2015,,0,,4c22ba361214f54a...@111.111.111.111,,,1800111111,6418589779977,9/10/2015,1:47:42,547,0.4741

201510-2218103e-6e27-11e5-9ae6-8bf952d26c75,2218103e-6e27-11e5-9ae6-8bf952d26c75,18583783788,Front Desk,18583783788,Front Desk,255,244,63611574460,NORMAL_CLEARING,1f23d1f4070e57a3...@111.111.111.111,a57c3b75aebe01e43cf2a1aa451fefe1,user_...@1000114.tenant.co,018583...@111.111.111.111,outbound,user_...@1000114.tenant.co,251eb2854a3037cac97a274be0021b6a,0,user_9t3cn0,18583783788,9/10/2015 1:47:00,1444355260,"Fri, 09 Oct 2015 01:47:40 GMT",9/10/2015,,0,,,,,,,,,,

201510-1f23d1f4070e...@111.111.111.111,1f23d1f4070e57a3...@111.111.111.111,18583783788,Front Desk,18583783788,Front Desk,264,263,63611574460,NORMAL_CLEARING,2218103e-6e27-11e5-9ae6-8bf952d26c75,,18001...@222.222.222.222:5060,018583...@111.111.111.111,inbound,18001...@222.222.222.222,,0,1800111111,18583783788,9/10/2015 1:47:00,1444355260,"Fri, 09 Oct 2015 01:47:40 GMT",9/10/2015,,0,,1f23d1f4070e57a3...@111.111.111.111,,,1800111111,6418583783788,9/10/2015,1:43:17,263,0.2279

201510-5683c0ea-6e25-11e5-bcc4-8bf952d26c75,5683c0ea-6e25-11e5-bcc4-8bf952d26c75,18581430056,18581430056,18581430056,18581430056,458,445,63611573892,NORMAL_CLEARING,2ad055da425a836f...@111.111.111.111,a57c3b75aebe01e43cf2a1aa451fefe1,user_...@1000114.tenant.co,018581...@111.111.111.111,outbound,user_...@1000114.tenant.co,251eb2854a3037cac97a274be0021b6a,0,user_9t3cn0,18581430056,9/10/2015 1:38:00,1444354692,"Fri, 09 Oct 2015 01:38:12 GMT",9/10/2015,,0,,,,,,,,,,

201510-2ad055da425a...@111.111.111.111,2ad055da425a836f...@111.111.111.111,18581430056,18581430056,18581430056,18581430056,469,468,63611573892,NORMAL_CLEARING,5683c0ea-6e25-11e5-bcc4-8bf952d26c75,,18001...@222.222.222.222:5060,018581...@111.111.111.111,inbound,18001...@222.222.222.222,,0,1800111111,18581430056,9/10/2015 1:38:00,1444354692,"Fri, 09 Oct 2015 01:38:12 GMT",9/10/2015,,0,,2ad055da425a836f...@111.111.111.111,,,1800111111,18581430056,9/10/2015,1:30:24,468,1.638

What gets populated is:

+----+------------------------------------------+----------+---------+-----------+-------------+----------+---------+---------+-------------+---------------------+-------------+---------------------+-------------+----------+------------------+--------+------------+---------------------+---------------------+---------------------+
| id | cgrid                                    | runid    | reqtype | direction | tenant      | category | account | subject | destination | setup_time          | pdd         | answer_time         | usage       | supplier | disconnect_cause | cost   | extra_info | created_at          | updated_at          | deleted_at          |
+----+------------------------------------------+----------+---------+-----------+-------------+----------+---------+---------+-------------+---------------------+-------------+---------------------+-------------+----------+------------------+--------+------------+---------------------+---------------------+---------------------+
|  1 | 95c7e5ccf93f207c64150b0b0b74bfb184c48cb9 | *default | *rated  | *out      | cgrates.org | call     |         |         |             | 0001-01-01 00:00:00 | 0.000000000 | 0001-01-01 00:00:00 | 0.000000000 |          |                  | 0.0000 |            | 2015-10-14 07:28:56 | 2015-10-14 07:28:56 | 0000-00-00 00:00:00 |
+----+------------------------------------------+----------+---------+-----------+-------------+----------+---------+---------+-------------+---------------------+-------------+---------------------+-------------+----------+------------------+--------+------------+---------------------+---------------------+---------------------+

DanB

unread,

Oct 15, 2015, 3:44:36 AM10/15/15

to cgr...@googlegroups.com

Hi Te,

Is there any chance you can post me your full cgrates.json configuration? Based on what you have shown here your CDRs are not reaching your CDRS component. Are you sure that you have posted full syslog?

DanB

PS:

--
You received this message because you are subscribed to the Google Groups "CGRateS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cgrates+u...@googlegroups.com.
To post to this group, send email to cgr...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cgrates/10cd9fa1-d560-4207-bd5b-d2f408afb639%40googlegroups.com.

Te Matau

unread,

Oct 16, 2015, 1:09:45 AM10/16/15

to CGRateS

{

// CGRateS Configuration file

//

// Used for cgradmin

// Starts rater, scheduler

"listen": {

"rpc_json": ":2012", // RPC JSON listening address

"rpc_gob": ":2013", // RPC GOB listening address

"http": ":2080", // HTTP listening address

},

"rater": {

"enabled": true, // enable Rater service: <true|false>

"cdrstats": "internal", // address where to reach the cdrstats service, empty to disable stats functionality<""|internal|x.y.z.y:1234>

"pubsubs": "internal", // address where to reach the pubusb service, empty to disable pubsub functionality: <""|internal|x.y.z.y:1234>

"users": "internal", // address where to reach the user service, empty to disable user profile functionality: <""|internal|x.y.z.y:1234>

},

"scheduler": {

"enabled": true, // start Scheduler service: <true|false>

},

"cdrs": {

"enabled": true, // start the CDR Server service: <true|false>

"rater": "internal", // address where to reach the Rater for cost calculation, empty to disable functionality: <""|internal|x.y.z.y:1234>

"cdrstats": "internal", // address where to reach the cdrstats service, empty to disable stats functionality<""|internal|x.y.z.y:1234>

},

"cdrstats": {

"enabled": true, // starts the cdrstats service: <true|false>

},

"pubsubs": {

"enabled": true, // starts PubSub service: <true|false>.

},

"users": {

"enabled": true, // starts User service: <true|false>.

"indexes": ["Uuid"], // user profile field indexes

},

"cdrc": {

"CDRC-CSV2": {

"enabled": true, // enable CDR client functionality

"cdr_in_dir": "/etc/cgrates/test/in", // absolute path towards the directory where the CDRs are stored

"cdr_out_dir": "/etc/cgrates/test/out", // absolute path towards the directory where processed CDRs will be moved

"cdr_source_id": "fs_csv", // free form field, tag identifying the source of the CDRs within CDRS database

"content_fields":[ // import template, tag will match internally CDR field, in case of .csv value will be represented by index of the field value