data source from a directory of files
32 views
Skip to first unread message
Mahesh Venkat
Nov 11, 2019, 2:50:16 AM11/11/19
to Metatron Discovery User Group
Hi,
How do I specify a datasource as a directory of json files ? Is there is a CLI to ingest data?
In a regular ingestion specs, I can specify it a "local" firehose configuration:
"firehose": {
"type": "local",
"baseDir": "examples/indexing/",
"filter": "wikipedia_index_data*"
}
"ioConfig": {
"type": "index_parallel",
"firehose": {
"type": "local",
"baseDir": "/Users/mvenkat/Desktop/Deals/data",
"filter": "*.json",
"parser": null
},
"appendToExisting": false
},
Reply all
Reply to author
Forward
0 new messages