Camel route for metacard ingest

[leon]

Hello,

I am trying to create a Camel route to ingest into the catalog. For the moment, I'm trying to perform everything using Blueprint XMLs. Starting simple, I'm using the following route, and I've been throwing geojsons into the monitored directory:

  <route>

    <from uri="file:/tmp/"/>

    <to uri="catalog:inputtransformer?mimeType=application/json&amp;id=geojson"/>

    <setHeader headerName="operation">

      <constant>CREATE</constant>

    </setHeader>

    <to uri="catalog:framework"/>

  </route>

 

Everything runs fine up to the pre-ingest security filter - I've attached a log excerpt below. Uninstalling catalog-security-operationplugin progresses this further, but an org.apache.shiro.UnavailableSecurityManagerException is thrown later. 

From what I understand, I need to bind a reference to either the system subject or the guest subject to the exchange for this to work. Is there a means to do this using pure Blueprint XML?

Thanks in advance,

Leon

---

04:49:13,716 | DEBUG | ile:///tmp/ | camel.component.file.GenericFileConsumer  188 | che.camel.camel-core | Total 1 files to consume
04:49:13,716 | DEBUG | ile:///tmp/ | camel.component.file.GenericFileConsumer  451 | che.camel.camel-core | About to process file: GenericFile[/tmp/tst2.json] using exchange: Exchange[]
04:49:13,717 | DEBUG | ile:///tmp/ | amel.component.file.GenericFileConverter  179 | che.camel.camel-core | Read file /tmp/tst2.json (no charset)
04:49:13,717 | DEBUG | ile:///tmp/ | org.apache.camel.processor.SendProcessor  141 | che.camel.camel-core | >>>> log://ddf Exchange[ID-ddf-ddfinternal-dom-34207-1507587362264-22-467]
04:49:13,717 | DEBUG | ile:///tmp/ | org.apache.camel.processor.SendProcessor  141 | che.camel.camel-core | >>>> catalog://inputtransformer?id=geojson&mimeType=application%2Fjson Exchange[ID-ddf-ddfinternal-dom-34207-1507587362264-22-467]
04:49:13,717 | DEBUG | ile:///tmp/ | .catalog.transformer.TransformerProducer   69 | -core-camelcomponent | ENTERING: process
04:49:13,717 | DEBUG | ile:///tmp/ | .catalog.transformer.TransformerProducer   71 | -core-camelcomponent | exchange pattern = InOnly
04:49:13,717 | DEBUG | ile:///tmp/ | .catalog.transformer.TransformerProducer   87 | -core-camelcomponent | transformerId retrieved from CamelCatalogEndpoint
04:49:13,717 | DEBUG | ile:///tmp/ | .catalog.transformer.TransformerProducer   91 | -core-camelcomponent | transformerId = geojson
04:49:13,718 | DEBUG | ile:///tmp/ | .catalog.transformer.TransformerProducer   96 | -core-camelcomponent | MIME Type retrieved from CamelCatalogEndpoint
04:49:13,718 | DEBUG | ile:///tmp/ | .catalog.transformer.TransformerProducer  100 | -core-camelcomponent | MIME Type = application/json
04:49:13,718 | DEBUG | ile:///tmp/ | .catalog.transformer.TransformerProducer  106 | -core-camelcomponent | Got a MimeTypeToTransformerMapper service
04:49:13,718 | DEBUG | ile:///tmp/ | nputtransformer.InputTransformerProducer  107 | -core-camelcomponent | List of matches for mimeType [application/json; id=geojson]: [InputTransformer {Impl=ddf.catalog.transformer.input.geojson.GeoJsonInputTransformer, id=geojson, mime-type=application/json}]
04:49:13,718 | DEBUG | ile:///tmp/ | er.input.geojson.GeoJsonInputTransformer  125 | on-input-transformer | Found registered MetacardType: ddf.metacard
04:49:13,719 | DEBUG | ile:///tmp/ | er.input.geojson.GeoJsonInputTransformer  130 | on-input-transformer | Metacard type name: ddf.metacard
04:49:13,719 | DEBUG | ile:///tmp/ | .catalog.transformer.TransformerProducer  119 | -core-camelcomponent | EXITING: process
04:49:13,719 | DEBUG | ile:///tmp/ | org.apache.camel.processor.SendProcessor  141 | che.camel.camel-core | >>>> log://ddfmc Exchange[ID-ddf-ddfinternal-dom-34207-1507587362264-22-467]
04:49:13,720 | DEBUG | ile:///tmp/ | org.apache.camel.processor.SendProcessor  141 | che.camel.camel-core | >>>> catalog://framework Exchange[ID-ddf-ddfinternal-dom-34207-1507587362264-22-467]
04:49:13,720 | DEBUG | ile:///tmp/ | nent.catalog.framework.FrameworkProducer  122 | -core-camelcomponent | Entering process method
04:49:13,720 | DEBUG | ile:///tmp/ | nent.catalog.framework.FrameworkProducer  552 | -core-camelcomponent | Reading in body data as Metacard...
04:49:13,720 | DEBUG | ile:///tmp/ | nent.catalog.framework.FrameworkProducer  558 | -core-camelcomponent | Successfully read in body data as Metacard 
04:49:13,720 | DEBUG | ile:///tmp/ | nent.catalog.framework.FrameworkProducer  193 | -core-camelcomponent | Validation of Metacard list passed...
04:49:13,721 | DEBUG | ile:///tmp/ | nent.catalog.framework.FrameworkProducer  204 | -core-camelcomponent | Making CREATE call to Catalog Framework...
04:49:13,721 | DEBUG | ile:///tmp/ | catalog.impl.operations.SourceOperations  267 | re-standardframework | Checking if source "ddf.distribution" is available...
04:49:13,721 | DEBUG | ile:///tmp/ | talog.plugin.clientinfo.ClientInfoPlugin  105 | og-plugin-clientinfo | No client info was stored for this thread [Camel (simple) thread #34 - file:///tmp/]
04:49:13,721 | DEBUG | ile:///tmp/ | ddf.catalog.data.impl.MetacardImpl        563 | core-injectattribute | Attribute resource-uri was not found, returning null
04:49:13,722 | DEBUG | ile:///tmp/ | f.catalog.security.plugin.SecurityPlugin  125 | alog-security-plugin | No security subject found, cannot add to current operation.
04:49:13,724 | DEBUG | ile:///tmp/ | nent.catalog.framework.FrameworkProducer  166 | -core-camelcomponent | Exception cataloging metacards
ddf.catalog.source.IngestException: Error during pre-ingest:
at ddf.catalog.impl.operations.CreateOperations.doCreate(CreateOperations.java:254)[349:catalog-core-standardframework:2.10.3]
at ddf.catalog.impl.operations.CreateOperations.create(CreateOperations.java:128)[349:catalog-core-standardframework:2.10.3]
at ddf.catalog.impl.CatalogFrameworkImpl.create(CatalogFrameworkImpl.java:244)[349:catalog-core-standardframework:2.10.3]
at Proxyfbc9df24_fd73_4120_95a5_44955358ae38.create(Unknown Source)[:]
at ddf.camel.component.catalog.framework.FrameworkProducer.create(FrameworkProducer.java:205)[336:catalog-core-camelcomponent:2.10.3]
at ddf.camel.component.catalog.framework.FrameworkProducer.process(FrameworkProducer.java:139)[336:catalog-core-camelcomponent:2.10.3]
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:145)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:542)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:120)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:460)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:227)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:191)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)[175:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)[175:org.apache.camel.camel-core:2.18.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_144]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)[:1.8.0_144]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)[:1.8.0_144]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)[:1.8.0_144]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_144]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_144]
at java.lang.Thread.run(Thread.java:748)[:1.8.0_144]
Caused by: ddf.catalog.plugin.StopProcessingException: Unable to filter contents of current message, no user Subject available.
at ddf.catalog.security.filter.plugin.FilterPlugin.getSubject(FilterPlugin.java:354)[463:catalog-security-filter:2.10.3]
at ddf.catalog.security.filter.plugin.FilterPlugin.processPreCreate(FilterPlugin.java:91)[463:catalog-security-filter:2.10.3]
at ddf.catalog.impl.operations.CreateOperations.processPrecreateAccessPlugins(CreateOperations.java:594)[349:catalog-core-standardframework:2.10.3]
at ddf.catalog.impl.operations.CreateOperations.doCreate(CreateOperations.java:236)[349:catalog-core-standardframework:2.10.3]

... 25 more

[Scott Tustison]

You are correct that you need to create or acquire a subject in order to ingest files from your own custom camel route. I would suggest using the guest subject as that has a reduced set of privileges if you have no other means of identifying the client. There is not a blueprint only way to attach a subject to the route, but you can do so with a custom bean in your camel route. We have several examples of this within DDF already, one of which is the Content Directory Monitor, which appears as though it would already suit your needs, if you are just attempting to ingest files from a particular directory.

Scott