Trying to use INPUT_LINE_FILTER

[Doug]

Hi,

I am trying to use INPUT_LINE_FILTER in a loading job and am getting the error "Pre-defined filters can be ONLY used in reject_line_rule!"

This doesn't make sense as (I assume) the definition of the loading job is like a software function and so anything defined in it is not a global thing.

I was trying to first define the filter -- I could not find any examples of the use of this at all, which was annoying - and so worked it out, I think.

i.e.  DEFINE INPUT_LINE_FILTER filterName = ($"field1" != $"field2") ;

(the document should state quotes or something should not surround the contents of the filter)

But then trying to use it as in USING REJECT_LINE_RULE=filterName, the CREATE LOADING JOB would break with the error above.  When I comment out the DEFINE line, then it compiles. This implies the filter has been set and is never unset. And I can't seem to delete it (if this is the case)

So I just am not entirely confident the job will do what I expect it to.

And the documentation should indicate if the filterName should be in quotes or not.  A simple of example of using this feature would solve all these issues.

P.S. the text based help file "help/gsql/gsql_help_basic.txt" talks about "REJECT_LIST_RULE" which is incorrect...

thanks. I am transferring our system from Neo4j and am happy with things so far. Neo4j refused to load the amount of data I need to deal with (not that much, I thought) without 32GB (or more!) of RAM... and it was taking forever to load a subset.  Tigergraph is doing everything in under 5 minutes.

PSS: I searched the examples on github and found an example, which matches exactly what I was doing.  So my issue still is present.  

The PDF cheatsheet also is more detailed and shows what I'd expect to see.  I have been predominantly using the docs.tigergraph.com website.

Doug

[Doug]

Ok, I figured it out.  My filter was actually using gsql_substring($"field1",0,10) != $"field2" and that was the problem -- no functions at all allowed in filters.  This was not immediately clear in the docs.

Doug