tl;dr: We use dask to accelerate parameter searches over machine learning pipelines by naming consistently. Read dask sklearn dasklearn Wed 16 September 2015
Analyzing 1.7 Billion Reddit Comments with Blaze and Impala by Daniel Rodriguez and Kristopher Overholt Blaze is a Python library and interface to query data on different storage systems. Blaze works by translating a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze gives Python users a familiar interface to query data living in other data storage systems such as SQL databases, NoSQL data stores, Spark, Hive, Impala, and raw data files such as CSV, JSON, and HDF5. Hive Read blaze impala hive reddit Tue 08 September 2015
Analyzing Reddit Comments with Dask and Castra by Jim Crist The scientific Python ecosystem is great for doing data analysis. Packages like NumPy and Pandas provide an excellent interface to doing complicated computations on datasets. With only a few lines of code one can load some data into a Pandas DataFrame, run some analysis, and generate a plot of the results. However, this workflow starts to falter when working with data that's larger than the RAM on your computer. At this point people often move their workflow from a Python based one into some other larger system like Spark or Hadoop. These are great at what they do, but for small problems are a bit overkill Read dask castra reddit
Talks and Tutorials
Agda, BlogLiterately, BluePrintCSS, DiscussionSupportSystem, Elm, HFitUI, MFlow, Nomyx-Core, Nomyx-Web, Spock, accelerate, ace, acme-everything, alerts, anatomy, apiary, apiary-clientsession, apiary-cookie, apiary-persistent, ats-format, blaze, blaze-bootstrap, blaze-colonnade, blaze-html-contrib, blaze-html-hexpat, blaze-html-truncate, blaze-htmx, blaze-shields, blaze-svg, blazeMarker, blazeT, blizzard-html, canteven-template, cheapskate, cheapskate-highlight, cheapskate-lucid, clckwrks, clckwrks-plugin-bugs, clckwrks-plugin-ircbot, clckwrks-plugin-media, cmark-highlight, curryer, dataflow, diagrams-svg, digestive-bootstrap, digestive-functors, digestive-functors-blaze, dingo-core, dingo-widgets, dixi, doctemplates, dom-selector, elm-compiler, ema, emanote, espial, esqueleto, eventlog2html, fay, firefly, formlets, fpco-api, front, futhark, geni-util, ghclive, gitit, gmail-simple, hablog, haggis, hakyll, hakyll-blaze-templates, hakyll-contrib-csv, hakyll-elm, hakyll-shakespeare, hakyll-shortcode, hamlet, happstack-authenticate, happstack-server, haskyapi, hatex-guide, hax, heckle, heist, heist-emanote, hermes, heterocephalus, highlighter, highlighter2, highlighting-kate, hledger-web, homplexity, hoogle, hyper, idris, ihaskell-blaze, ihaskell-inline-r, ihaskell-rlangqq, ihp-hsx, imm, imprevu-happstack, inliterate, jmacro-rpc, jmacro-rpc-happstack, knit-haskell, lambdacms-core, lapack, layout-bootstrap, leksah, lightning-haskell, llvm-tools, lmonad-yesod, mailtrap, mandrill, markdown, markdown-kate, markup, mig, mig-extra, mig-server, mig-swagger-ui, mmark-ext, myxine-client, named-formlet, nested-routes, newsletter, nirum, nomyx-core, nomyx-web, notmuch-web, pandoc, pandoc-filter-indent, parochial, persistent, persistent-odbc, persistent-test, plotlyhs, processing, purescript, r3x-haskell-sdk, readme-lhs, reform-blaze, repo-based-blog, rest-gen, rfc, rfc-servant, saferoute, scholdoc, scotty-blaze, scotty-hastache, seacat, senza, servant-blaze, servant-static-th, ses-html, ses-html-snaplet, shakespeare, simple-css, simple-form, skell, skylighting, skylighting-core, skylighting-format-blaze-html, snap-app, snap-blaze, snap-blaze-clay, snap-extras, snaplet-ses-html, sockets-and-pipes, stackage-curator, stagen, stan, swagger-test, swarm, taggy, tasty-html, text-and-plots, tianbar, toodles, trifecta, uuagd, verismith, wai-app-file-cgi, wai-app-static, wai-devel, wai-middleware-auth, wai-middleware-content-type, web-page, webpage, webwire, xml-conduit, xml2html, xmlhtml, yeamer, yesod, yesod-alerts, yesod-angular-ui, yesod-articles, yesod-auth, yesod-auth-account, yesod-auth-account-fork, yesod-auth-bcrypt, yesod-auth-simple, yesod-bootstrap, yesod-colonnade, yesod-content-pdf, yesod-core, yesod-elements, yesod-form, yesod-form-bootstrap4, yesod-form-richtext, yesod-goodies, yesod-markdown, yesod-newsfeed, yesod-platform, yesod-rst, yesod-test, yesod-vend, yu-utils
2captcha, ADPfusion, ADPfusionForest, ADPfusionSet, AMI, Advise-me, AesonBson, AlignmentAlgorithms, Allure, BioHMM, Biobase, BiobaseBlast, BiobaseENA, BiobaseFR3D, BiobaseFasta, BiobaseHTTP, BiobaseInfernal, BiobaseTrainingData, BiobaseTurner, BiobaseTypes, BiobaseVienna, BiobaseXNA, BlastHTTP, BlogLiterately-diagrams, CMCompare, CarneadesDSL, CarneadesIntoDung, Chart-diagrams, Coadjute, ConcurrentUtils, DAV, DOH, DPM, DPutils, DRBG, DSA, Deadpan-DDP, DigitalOcean, DnaProteinAlignment, Dust, Dust-crypto, Dust-tools, EntrezHTTP, Extra, FAI, Facebook-Password-Hacker-Online-Latest-Version, Finance-Quote-Yahoo, Forestry, FormalGrammars, Frames-beam, Frames-map-reduce, Frames-streamly, GLM, Gene-CluEDO, GenussFold, Gleam, GoogleDirections, GoogleSuggest, GoogleTranslate, GrammarProducts, Graphalyze, HAppSHelpers, HROOT, HROOT-core, HROOT-graf, HROOT-hist, HROOT-io, HROOT-math, HROOT-net, HROOT-tree, HXMPP, HasChor, HaskellNet, HaskellNet-SSL, Hastodon, Hawk, Hermes, Hoed, HsHTSLib, HsWebots, HueAPI, Hydrogen, IPv6DB, LambdaHack, Lastik, LslPlus, Lykah, MC-Fold-DP, MIP, MIP-glpk, MailchimpSimple, MicrosoftTranslator, MusicBrainz, MutationOrder, NGLess, NTRU, NaturalLanguageAlphabets, Nussinov78, OGDF, OrchestrateDB, Ordinary, PUH-Project, PageIO, Paillier, PandocAgda, Parry, PrimitiveArray, PrimitiveArray-Pretty, Quelea, QuickPlot, RNAFold, RNAdesign, RNAdraw, RNAlien, RNAwolf, RSA, Redmine, ReplaceUmlaut, Rlang-QQ, SVD2HS, SVGFonts, SciBaseTypes, SciFlow, SciFlow-drmaa, ShortestPathProblems, Shpadoinkle, Shpadoinkle-backend-pardiff, Shpadoinkle-backend-snabbdom, Shpadoinkle-backend-static, Shpadoinkle-console, Shpadoinkle-debug, Shpadoinkle-developer-tools, Shpadoinkle-disembodied, Shpadoinkle-html, Shpadoinkle-isreal, Shpadoinkle-lens, Shpadoinkle-router, Shpadoinkle-streaming, Shpadoinkle-template, Shpadoinkle-widgets, SimpleServer, Slides, Spock-api-server, Spock-auth, Spock-core, Spock-digestive, Spock-lucid, Spock-worker, StockholmAlignment, SvgIcons, Twofish, URLT, Unixutils, VKHS, ViennaRNA-extras, Villefort, WMSigner, WebCont, Wheb, WordAlignment, XSaiga, abeson, abstract-par-accelerate, accelerate-arithmetic, accelerate-bignum, accelerate-blas, accelerate-cublas, accelerate-cuda, accelerate-cufft, accelerate-examples, accelerate-fft, accelerate-fftw, accelerate-fourier, accelerate-io, accelerate-io-JuicyPixels, accelerate-io-array, accelerate-io-bmp, accelerate-io-bytestring, accelerate-io-cereal, accelerate-io-repa, accelerate-io-serialise, accelerate-io-vector, accelerate-kullback-liebler, accelerate-llvm, accelerate-llvm-native, accelerate-llvm-ptx, accelerate-random, accelerate-typelits, accelerate-utility, access-token-provider, achille, acousticbrainz-client, advent-of-code-api, aeson-bson, aeson-injector, affection, afis, agda-language-server, agda-snippets, agda-snippets-hakyll, agda-unused, agentx, airbrake, airship, airtable-api, aivika-experiment-diagrams, alerta, alfred, algebraic, algebraic-graphs-io, algolia, ally-invest, alto, amazon-emailer-client-snap, amazon-products, amazonka, amazonka-accessanalyzer, amazonka-account, amazonka-alexa-business, amazonka-amp, amazonka-amplify, amazonka-amplifybackend, amazonka-amplifyuibuilder, amazonka-apigateway, amazonka-apigatewaymanagementapi, amazonka-apigatewayv2, amazonka-appconfig, amazonka-appconfigdata, amazonka-appflow, amazonka-appintegrations, amazonka-application-autoscaling, amazonka-application-insights, amazonka-applicationcostprofiler, amazonka-appmesh, amazonka-apprunner, amazonka-appstream, amazonka-appsync, amazonka-arc-zonal-shift, amazonka-athena, amazonka-auditmanager, amazonka-autoscaling, amazonka-autoscaling-plans, amazonka-backup, amazonka-backup-gateway, amazonka-backupstorage, amazonka-batch, amazonka-billingconductor, amazonka-braket, amazonka-budgets, amazonka-certificatemanager, amazonka-certificatemanager-pca, amazonka-chime, amazonka-chime-sdk-identity, amazonka-chime-sdk-media-pipelines, amazonka-chime-sdk-meetings, amazonka-chime-sdk-messaging, amazonka-chime-sdk-voice, amazonka-cloud9, amazonka-cloudcontrol, amazonka-clouddirectory, amazonka-cloudformation, amazonka-cloudfront, amazonka-cloudhsm, amazonka-cloudhsmv2, amazonka-cloudsearch, amazonka-cloudsearch-domains, amazonka-cloudtrail, amazonka-cloudwatch, amazonka-cloudwatch-events, amazonka-cloudwatch-logs, amazonka-codeartifact, amazonka-codebuild, amazonka-codecommit, amazonka-codedeploy, amazonka-codeguru-reviewer, amazonka-codeguruprofiler, amazonka-codepipeline, amazonka-codestar, amazonka-codestar-connections, amazonka-codestar-notifications, amazonka-cognito-identity, amazonka-cognito-idp, amazonka-cognito-sync, amazonka-comprehend, amazonka-comprehendmedical, amazonka-compute-optimizer, amazonka-config, amazonka-connect, amazonka-connect-contact-lens, amazonka-connectcampaigns, amazonka-connectcases, amazonka-connectparticipant, amazonka-contrib-rds-utils, amazonka-controltower, amazonka-core, amazonka-cost-explorer, amazonka-cur, amazonka-customer-profiles, amazonka-databrew, amazonka-dataexchange, amazonka-datapipeline, amazonka-datasync, amazonka-detective, amazonka-devicefarm, amazonka-devops-guru, amazonka-directconnect, amazonka-discovery, amazonka-dlm, amazonka-dms, amazonka-docdb, amazonka-docdb-elastic, amazonka-drs, amazonka-ds, amazonka-dynamodb, amazonka-dynamodb-dax, amazonka-dynamodb-streams, amazonka-ebs, amazonka-ec2, amazonka-ec2-instance-connect, amazonka-ecr, amazonka-ecr-public, amazonka-ecs, amazonka-efs, amazonka-eks, amazonka-elastic-inference, amazonka-elasticache, amazonka-elasticbeanstalk, amazonka-elasticsearch, amazonka-elastictranscoder, amazonka-elb, amazonka-elbv2, amazonka-emr, amazonka-emr-containers, amazonka-emr-serverless, amazonka-evidently, amazonka-finspace, amazonka-finspace-data, amazonka-fis, amazonka-fms, amazonka-forecast, amazonka-forecastquery, amazonka-frauddetector, amazonka-fsx, amazonka-gamelift, amazonka-gamesparks, amazonka-glacier, amazonka-globalaccelerator, amazonka-glue, amazonka-grafana, amazonka-greengrass, amazonka-greengrassv2, amazonka-groundstation, amazonka-guardduty, amazonka-health, amazonka-healthlake, amazonka-honeycode, amazonka-iam, amazonka-identitystore, amazonka-imagebuilder, amazonka-importexport, amazonka-inspector, amazonka-inspector2, amazonka-iot, amazonka-iot-analytics, amazonka-iot-dataplane, amazonka-iot-jobs-dataplane, amazonka-iot-roborunner, amazonka-iot1click-devices, amazonka-iot1click-projects, amazonka-iotdeviceadvisor, amazonka-iotevents, amazonka-iotevents-data, amazonka-iotfleethub, amazonka-iotfleetwise, amazonka-iotsecuretunneling, amazonka-iotsitewise, amazonka-iotthingsgraph, amazonka-iottwinmaker, amazonka-iotwireless, amazonka-ivs, amazonka-ivschat, amazonka-kafka, amazonka-kafkaconnect, amazonka-kendra, amazonka-keyspaces, amazonka-kinesis, amazonka-kinesis-analytics, amazonka-kinesis-firehose, amazonka-kinesis-video, amazonka-kinesis-video-archived-media, amazonka-kinesis-video-media, amazonka-kinesis-video-signaling, amazonka-kinesis-video-webrtc-storage, amazonka-kinesisanalyticsv2, amazonka-kms, amazonka-lakeformation, amazonka-lambda, amazonka-lex-models, amazonka-lex-runtime, amazonka-lexv2-models, amazonka-license-manager, amazonka-license-manager-linux-subscriptions, amazonka-license-manager-user-subscriptions, amazonka-lightsail, amazonka-location, amazonka-lookoutequipment, amazonka-lookoutmetrics, amazonka-lookoutvision, amazonka-m2, amazonka-macie, amazonka-maciev2, amazonka-managedblockchain, amazonka-marketplace-analytics, amazonka-marketplace-catalog, amazonka-marketplace-entitlement, amazonka-marketplace-metering, amazonka-mechanicalturk, amazonka-mediaconnect, amazonka-mediaconvert, amazonka-medialive, amazonka-mediapackage, amazonka-mediapackage-vod, amazonka-mediastore, amazonka-mediastore-dataplane, amazonka-mediatailor, amazonka-memorydb, amazonka-mgn, amazonka-migration-hub-refactor-spaces, amazonka-migrationhub, amazonka-migrationhub-config, amazonka-migrationhuborchestrator, amazonka-migrationhubstrategy, amazonka-ml, amazonka-mobile, amazonka-mq, amazonka-mtl, amazonka-mwaa, amazonka-neptune, amazonka-network-firewall, amazonka-networkmanager, amazonka-nimble, amazonka-oam, amazonka-omics, amazonka-opensearch, amazonka-opensearchserverless, amazonka-opsworks, amazonka-opsworks-cm, amazonka-organizations, amazonka-outposts, amazonka-panorama, amazonka-personalize, amazonka-personalize-events, amazonka-personalize-runtime, amazonka-pi, amazonka-pinpoint, amazonka-pinpoint-email, amazonka-pinpoint-sms-voice, amazonka-pinpoint-sms-voice-v2, amazonka-pipes, amazonka-polly, amazonka-pricing, amazonka-privatenetworks, amazonka-proton, amazonka-qldb, amazonka-qldb-session, amazonka-quicksight, amazonka-ram, amazonka-rbin, amazonka-rds, amazonka-rds-data, amazonka-redshift, amazonka-redshift-data, amazonka-redshift-serverless, amazonka-rekognition, amazonka-resiliencehub, amazonka-resource-explorer-v2, amazonka-resourcegroups, amazonka-resourcegroupstagging, amazonka-robomaker, amazonka-rolesanywhere, amazonka-route53, amazonka-route53-autonaming, amazonka-route53-domains, amazonka-route53-recovery-cluster, amazonka-route53-recovery-control-config, amazonka-route53-recovery-readiness, amazonka-route53resolver, amazonka-rum, amazonka-s3, amazonka-s3-encryption, amazonka-s3-streaming, amazonka-s3outposts, amazonka-sagemaker, amazonka-sagemaker-a2i-runtime, amazonka-sagemaker-edge, amazonka-sagemaker-featurestore-runtime, amazonka-sagemaker-geospatial, amazonka-sagemaker-metrics, amazonka-sagemaker-runtime, amazonka-savingsplans, amazonka-scheduler, amazonka-schemas, amazonka-sdb, amazonka-secretsmanager, amazonka-securityhub, amazonka-securitylake, amazonka-serverlessrepo, amazonka-service-quotas, amazonka-servicecatalog, amazonka-servicecatalog-appregistry, amazonka-ses, amazonka-sesv2, amazonka-shield, amazonka-signer, amazonka-simspaceweaver, amazonka-sms, amazonka-sms-voice, amazonka-snow-device-management, amazonka-snowball, amazonka-sns, amazonka-sqs, amazonka-ssm, amazonka-ssm-contacts, amazonka-ssm-incidents, amazonka-ssm-sap, amazonka-sso, amazonka-sso-admin, amazonka-sso-oidc, amazonka-stepfunctions, amazonka-storagegateway, amazonka-sts, amazonka-support, amazonka-support-app, amazonka-swf, amazonka-synthetics, amazonka-test, amazonka-textract, amazonka-timestream-query, amazonka-timestream-write, amazonka-transcribe, amazonka-transfer, amazonka-translate, amazonka-voice-id, amazonka-waf, amazonka-waf-regional, amazonka-wafv2, amazonka-wellarchitected, amazonka-wisdom, amazonka-workdocs, amazonka-worklink, amazonka-workmail, amazonka-workmailmessageflow, amazonka-workspaces, amazonka-workspaces-web, amazonka-xray, amby, ampersand, amqp, amqp-conduit, amqp-streamly, amqp-worker, analyze-client, anansi-pandoc, animate-frames, annah, antagonist, antigate, antiope-athena, antiope-contract, antiope-core, antiope-dynamodb, antiope-es, antiope-messages, antiope-optparse-applicative, antiope-s3, antiope-shell, antiope-sns, antiope-sqs, antiope-swf, apecs-gloss, apecs-physics, apecs-physics-gloss, api-builder, api-maker, api-monobank, api-rpc-accumulate, api-rpc-factom, api-rpc-pegnet, api-yoti, apiary-authenticate, apiary-eventsource, apiary-helics, apiary-http-client, apiary-logger, apiary-memcached, apiary-mongoDB, apiary-purescript, apiary-redis, apiary-session, apiary-websockets, apioiaf-client, apis, apns-http2, apotiki, appc, appendful-persistent, approveapi, arbor-postgres, arch-hs, arch-web, archive, archive-tar-bytestring, arrowp-qq, asana, asap, assumpta, atlas, atlassian-connect-core, atndapi, atom-conduit, ats-pkg, ats-setup, attoparsec-data, aur, aur-api, aura, authenticate, authenticate-oauth, authoring, autodocodec-servant-multipart, avahi, avers, avers-api, avers-api-docs, avers-server, aviation-cessna172-diagrams, avro, avro-piper, aws, aws-cloudfront-signer, aws-configuration-tools, aws-dynamodb-conduit, aws-dynamodb-streams, aws-easy, aws-ec2, aws-elastic-transcoder, aws-general, aws-kinesis, aws-kinesis-client, aws-kinesis-reshard, aws-lambda, aws-lambda-haskell-runtime, aws-lambda-haskell-runtime-wai, aws-larpi, aws-performance-tests, aws-route53, aws-sdk, aws-sdk-xml-unordered, aws-ses-easy, aws-sign4, aws-simple, aws-sns, aws-sns-verify, aws-transcribe-ws, aws-xray-client-persistent, axel, axiom, axiomatic-classes, azimuth-hs, azure-acs, azure-email, azure-functions-worker, azure-service-api, azure-servicebus, azurify, b9, backblaze-b2-hs, bake, ballast, bamboo, bamboo-plugin-highlight, bamboo-plugin-photo, bamboo-theme-blueprint, bamboo-theme-mini-html5, barchart, barrier, base58address, basex-client, batchd-core, batchd-docker, batchd-libvirt, battlenet, battlenet-yesod, battleplace, battleplace-api, battleships, bcp47, bcp47-orphans, bcrypt, bdcs, bdcs-api, beam-automigrate, beam-migrate, beam-newtype-field, beam-postgres, beam-sqlite, beeminder-api, belka, bench-graph, bench-show, bet, bidi-icu, bimap-server, binance-exports, bioinformatics-toolkit, bip32, birch-beer, bird, biscuit-servant, bitcoin-address, bitcoin-api, bitcoin-api-extra, bitcoin-block, bitcoin-compact-filters, bitcoin-keys, bitcoin-payment-channel, bitcoin-scripting, bitcoin-tx, bitcoind-regtest, bitcoind-rpc, bittorrent, bittrex, bitx-bitcoin, bkr, blacktip, blagda, blank-canvas, ble, blockfrost-api, blockfrost-client, blockfrost-client-core, blockfrost-pretty, blogination, bloodhound, bloodhound-amazonka-auth, bloomfilter-redis, blunt, bnb-staking-csvs, bodhi, bond, bond-haskell, bond-haskell-compiler, boots-cloud, boots-web, borel, box-socket, braid, brick-skylighting, brok, bronyradiogermany-streaming, browscap, bson, bson-generic, bson-generics, bson-lens, bson-mapping, btc-lsp, btree-concurrent, buchhaltung, bugsnag, bugsnag-haskell, bugsnag-wai, bugsnag-yesod, bugzilla, bugzilla-redhat, bulmex, bureaucromancy, buttplug-hs-core, bytehash, bytestring-arbitrary, bytestring-typenats, c-mosquitto, cab, cabal-cache, cabal-debian, cabal-file, cabal-install, cabal2nix, cached-json-file, cachix, cachix-api, cake, calamity, call-alloy, campfire, canteven-http, capataz, captcha-2captcha, captcha-capmonster, captcha-core, carbonara, cas-hashable, cas-hashable-s3, cas-store, casa-abbreviations-and-acronyms, casa-client, casa-types, cassandra-cql, cayley-client, cdp, ceilometer-common, celtchar, cerberus, cereal-uuid, certificate, chakra, charade, chart-svg, chart-svg-various, chart-unit, charter, chatwork, cheapskate-terminal, checkmate, cherry-core-alpha, chevalier-common, chez-grater, chiasma, chiasma-test, chromatin, cicero-api, cielo, cipher-aes, cipher-aes128, circle, circlehs, cisco-spark-api, citation-resolve, citeproc, cj-token, cl3, cl3-hmatrix-interface, cl3-linear-interface, claferwiki, clang-pure, clarifai, clash-ghc, clash-lib, clash-lib-hedgehog, clash-shake, clash-systemverilog, clash-verilog, clash-vhdl, clashilator, classy-influxdb-simple, classy-miso, classy-prelude-conduit, classy-prelude-yesod, clckwrks-cli, clckwrks-plugin-mailinglist, clckwrks-plugin-page, clckwrks-plugin-redirect, clckwrks-theme-bootstrap, clckwrks-theme-clckwrks, clckwrks-theme-geo-bootstrap, clerk, cleveland, clickhouse-haskell, clientsession, clit, closed, cloud-haskell, cloud-seeder, cloudfront-signer, cmake-syntax, cmv, cobot-io, codeforces-cli, codeworld-api, codex, coformat,