[erlang-questions] ets:select badarg question (R15B)

[Matthew Evans]

Hi,

I'm running a select with a continuation on a public ets table with approximately 20,000 records.

The original call to the select is: ets:select(TableName,[{'$1',[],['$_']}],RecordsToGet), then an ets:select/1 on the continuation.

The process that is spawned to walk the table is sitting in a recursive loop doing a select (grabbing about 150 records each time), processing the matching records (which includes possibly deleting some of these records), then sleeping for a few seconds. It takes about 900 seconds to walk through the table. As well as records getting deleted, new records are added to the table during this time.

Every now and then I get an exception in the process doing the select/1:

Error in process <0.16218.1> on node 'xxxxx@E039D7001180' with exit value: {badarg,[{ets,select,[{'table_E0:39:D7:00:11:80',301,6,<<0 bytes>>,[{p_MacEntry.......  (truncated here)

This is the code (all running in the same process that started the select) obviously the last ets:select is causing the crash:

start_aging1(AgeInterval,MaxAge,SelectData) ->

     start_aging1(AgeInterval,MaxAge,SelectData,0).

start_aging1(_AgeInterval,_MaxAge,'$end_of_table',RecsAged) ->

     gen_server:cast(?MODULE,{completed_aging_operation,RecsAged});

start_aging1(_AgeInterval,MaxAge,{Matches,'$end_of_table'},RecsAged) ->

     RecordsAgedCount = age_records(Matches,[],MaxAge*1000000,os:timestamp()),

     gen_server:cast(?MODULE,{completed_aging_operation,RecsAged+RecordsAgedCount});

start_aging1(AgeInterval,MaxAge,{Matches,Continuation},RecsAged) ->

     RecordsAgedCount = age_records(Matches,[],MaxAge*1000000,os:timestamp()),

     timer:sleep(AgeInterval),

     start_aging1(AgeInterval,MaxAge,ets:select(Continuation),RecsAged+RecordsAgedCount).

I'm not sure what could be the cause of the select getting a bad arg.

Any ideas?

Thanks

Matt                                          

  

[Sverker Eriksson]

What options are you using when the table is created?

/Sverker, Erlang/OTP

> ------------------------------------------------------------------------
>
> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions
>

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Matthew Evans]

Hi

These are the options

ets:new(StationIdTableName,[{keypos,2},named_table,public]),

The only odd things I can think of is that during the select records are getting deleted and added. Also, the select operation is run slowly (over 900 seconds)

Again, this isn't every time I get the crash.

Matt

> Date: Mon, 15 Oct 2012 11:49:14 +0200
> From: sverker....@ericsson.com
> To: mattev...@hotmail.com
> CC: erlang-q...@erlang.org
> Subject: Re: [erlang-questions] ets:select badarg question (R15B)

[Sverker Eriksson]

You need to use ets:safe_fixtable/2 to get a "sensible" result from
select/match with continuation on unordered tables. Otherwise you may
miss records, get double hits and (as in your case) get badarg if the
table has shrunken due to deleted record causing the continuation to
point out of table bounds. One could argue that '$end_of_table' would be
a more suitable result than badarg for this case.

I see that the documentation is not clear about this. It describes usage
of safe_fixtable together with first/1 and next/2 but it does not state
that it also is needed for select/match with continuation.
Single call select/match does not need safe_fixtable.

[Matthew Evans]

Many thanks,

I'll give that a go. I am however a little worried about having a safe_fixtable "active" on a table for 900 seconds so might need to be a little smarter.

Cheers

Matt

> Date: Mon, 15 Oct 2012 14:50:34 +0200

[Matthew Evans]

I'll rephrase that. This is my concern from the ets documentation for safe_fixtable/2:

Note that no deleted objects are actually removed from a fixed table until it has been released. If a process fixes a table but never releases it, the memory used by the deleted objects will never be freed. The performance of operations on the table will also degrade significantly.

Is there a potential problem on having a safe_fixtable "active" for long periods of time on a table having lots of inserts and deletes?

Thanks

Matt

________________________________
> From: mattevans23@@hotmail.com
> To: sverker....@ericsson.com
> Date: Mon, 5 Oct 012 0::4::1 -400
> CC: erlang-q...@erlang.org
> Subject: Re: [erlang-questions] ets:select badarg question (R5BB)

>
> Many thanks,
>
> I'll give that a go. I am however a little worried about having a

> safe_fixtable "active" on a table for 00 seconds so might need to be a
> little smarter.
>
> Cheers
>
> Matt
>
> > Date: Mon, 5 Oct 012 4::0::4 +200
> > From: sverker....@ericsson.com
> > To: mattevans23@@hotmail.com
> > CC: erlang-q...@erlang.org
> > Subject: Re: [erlang-questions] ets:select badarg question (R5BB)
> >
> > You need to use ets:safe_fixtable/ to get a "sensible" result from

> > select/match with continuation on unordered tables. Otherwise you may
> > miss records, get double hits and (as in your case) get badarg if the
> > table has shrunken due to deleted record causing the continuation to
> > point out of table bounds. One could argue that '$end_of_table' would be
> > a more suitable result than badarg for this case.
> >
> > I see that the documentation is not clear about this. It describes usage

> > of safe_fixtable together with first/ and next/ but it does not state

> > that it also is needed for select/match with continuation.
> > Single call select/match does not need safe_fixtable.
> >
> > /Sverker, Erlang/OTP
> >
> > Matthew Evans wrote:
> > > Hi
> > > These are the options
> > >
> > >

> > > ets:new(StationIdTableName,[{keypos,}},named_table,public]),

> > > The only odd things I can think of is that during the select
> records are getting deleted and added. Also, the select operation is

> run slowly (over 00 seconds)

> > > Again, this isn't every time I get the crash.
> > > Matt
> > >
> > >

> > >> Date: Mon, 5 Oct 012 1::9::4 +200
> > >> From: sverker....@ericsson.com
> > >> To: mattevans23@@hotmail.com
> > >> CC: erlang-q...@erlang.org
> > >> Subject: Re: [erlang-questions] ets:select badarg question (R5BB)

> > >>
> > >> What options are you using when the table is created?
> > >>
> > >> /Sverker, Erlang/OTP
> > >>
> > >> Matthew Evans wrote:
> > >>
> > >>> Hi,
> > >>> I'm running a select with a continuation on a public ets table

> with approximately 0,,00 records.

> > >>> The original call to the select is:

> ets:select(TableName,[{'$'',[],['$_']}],RecordsToGet), then an
> ets:select/ on the continuation.

> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> The process that is spawned to walk the table is sitting in a

> recursive loop doing a select (grabbing about 50 records each time),

> processing the matching records (which includes possibly deleting some

> of these records), then sleeping for a few seconds. It takes about 00

> seconds to walk through the table. As well as records getting deleted,
> new records are added to the table during this time.
> > >>> Every now and then I get an exception in the process doing the

> select/::
> > >>> Error in process <..6218..>> on node 'xxxxx@E39DD001180'' with
> exit value: {badarg,[{ets,select,[{'table_E::9::D::0::1::0'',01,,,,<<

> bytes>>,[{p_MacEntry....... (truncated here)
> > >>> This is the code (all running in the same process that started
> the select) obviously the last ets:select is causing the crash:
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>

> > >>> start_aging((AgeInterval,MaxAge,SelectData) ->
> > >>> start_aging((AgeInterval,MaxAge,SelectData,)).
> > >>> start_aging((_AgeInterval,_MaxAge,'$end_of_table',RecsAged) ->
> > >>> gen_server:cast(?MODULE,{completed_aging_operation,RecsAged});
> > >>> start_aging((_AgeInterval,MaxAge,{Matches,'$end_of_table'},RecsAged) ->
> > >>> RecordsAgedCount =
> age_records(Matches,[],MaxAge*000000,,os:timestamp()),
> > >>>
> gen_server:cast(?MODULE,{completed_aging_operation,RecsAged+RecordsAgedCount});
> > >>> start_aging((AgeInterval,MaxAge,{Matches,Continuation},RecsAged) ->
> > >>> RecordsAgedCount =
> age_records(Matches,[],MaxAge*000000,,os:timestamp()),
> > >>> timer:sleep(AgeInterval),
> > >>>
> start_aging((AgeInterval,MaxAge,ets:select(Continuation),RecsAged+RecordsAgedCount).

[Patrik Nyblom]

Hi!

On 10/15/2012 05:09 PM, Matthew Evans wrote:

I'll rephrase that. This is my concern from the ets documentation for safe_fixtable/2:

Note that no deleted objects are actually removed from a fixed table until it has been released. If a process fixes a table but never releases it, the memory used by the deleted objects will never be freed. The performance of operations on the table will also degrade significantly.

Is there a potential problem on having a safe_fixtable "active" for long periods of time on a table having lots of inserts and deletes?

It all depends...
Memory-wise the deletes will actually take place after the fixation is released.
Fixtable also means longer link chains in the buckets, so if you for example will replace all the objects in the table once during the timeslot, lookup will need to traverse twice as long lists for every element.
If you do heavy insertion (without deletion), you will also get much longer chains, as the number of buckets will not increase while the table is 'fixed'.

So it depends on how much 'lots' is in your case...

Have you considered using ordered_set instead? Then you wouldn't need fixation and would have no problem traversing the "moving" table. Performance on ordered set is not as bad as you would think and you can do nice optimizations when selecting by using partially bound keys.

/Patrik

[Sverker Eriksson]

Deleting a lot of objects in a fixed table may cause memory problems as
the deleted object are not deallocated. They are instead marked as
deleted and will be kept around until the table is unfixed.

Increasing the number of objects in a fixed table may cause performance
problems. The hash table will not grow, leading to long linear search
through chains of objects hashing to the same bucket.

/Sverker