Issue #421: Add mechanism to allow large files for ingest (blais/beancount)
12 views
Skip to first unread message
tbm13
Feb 25, 2020, 5:03:42 AM2/25/20
to bean...@googlegroups.com
New issue 421: Add mechanism to allow large files for ingest
https://bitbucket.org/blais/beancount/issues/421/add-mechanism-to-allow-large-files-for
Martin Michlmayr:
I have a 32 MB CSV file \(it contain all invalid transactions that have been attempted in addition to the valid ones that actually went through\). bean-extract doesn’t like this because:
```
# A file size beyond which we will simply ignore the file. This is used to skip
# large files that are commonly co-present in a Downloads directory.
FILE_TOO_LARGE_THRESHOLD = 810241024
```
Would it be possible to add an option for individual importers to override the size limit \(my importer copes with the 32 MB file just fine\) or does this have side effects \(caching, etc\)?
https://bitbucket.org/blais/beancount/issues/421/add-mechanism-to-allow-large-files-for
Martin Michlmayr:
I have a 32 MB CSV file \(it contain all invalid transactions that have been attempted in addition to the valid ones that actually went through\). bean-extract doesn’t like this because:
```
# A file size beyond which we will simply ignore the file. This is used to skip
# large files that are commonly co-present in a Downloads directory.
FILE_TOO_LARGE_THRESHOLD = 810241024
```
Would it be possible to add an option for individual importers to override the size limit \(my importer copes with the 32 MB file just fine\) or does this have side effects \(caching, etc\)?
Reply all
Reply to author
Forward
0 new messages