MNIST Structure/Debug question

[Efram Burlingame]

Hello,

Some background, I’m an ML learner and medium to low-end programmer.  So I’m not sure this is the correct place to ask this question, feel free to redirect me as appropriate.  I have been running through some examples on the S4TF IRIS example and expanding it to my own XOR version that pulls in randomly generated data (to prove I can import and format data correctly).  That all works great and the examples are much appreciated.  However when I jump off to the MNIST “next step” I am lost as to the construction and accessing of the MIST dataset.  I copied some code off the internet that shows the importing and training for the MNIST dataset that appears to work but its a little smoke and mirrors for me.  Does anyone know where I can find the documentation on how to access the MNIST data structures inside?  I would love to do something simple like pull out one image and access its array instances.  I can’t really tell how to do that.  I just tried something simple like “print( type(of: MNIST))” which gave a lot of interesting detail that I had no idea how to expand.  

I’m importing the MNIST dataset with:

Let mnist = MNIST(batchsize: batchsize)

So my big questions are:

1) How do I know what the structure is of the data in MNIST?

2) How to I appropriately access (i.e. what commands or grammar do I use to access sub-portions) the data?

I excited to run the MNIST version and hopeful for the direction of S4TF.  Thank you for any help you are able to render, even if its just to tell me this isn’t the write location to ask this kind of information.  Thanks!

-Efram 

P.s. thanks to the Rick Wierenga blog for much info

[Brad Larson]

Sorry about the complexity of the MNIST (and other dataset) internals. The datasets in our swift-models repository have undergone evolution over time to minimize repeated code and to develop an underlying shared base. That base is the Epochs API that was described in this presentation (slides). We're working on building out a guide for how the Epochs API works, so I apologize for the currently lacking documentation around this.

The MNIST dataset in swift-models has two key properties: training and validation. Those are both lazily-mapped sequences (so that processing of images is done on-demand and not all upfront) that allow for iteration over the training and validation splits within the MNIST dataset. The MNIST dataset is further complicated by the fact that some of its shared functionality has been extracted into MNISTDatasetHandler.swift for use in three different MNIST variants.

The fetchMNISTDataset() function downloads the MNIST dataset files if they aren't already present in a local cache, extracts and caches them, and then reads the binary files to produce an array of (image bytes, integer label) tuples. The makeMNISTBatch() function is used by the lazy mapping of the Epochs API to create batches on-demand from a shuffled dataset. In this case, it creates image and label Tensors from batches of (image bytes, integer label) tuples.

The MNIST dataset further conforms to ImageClassificationData to allow it to be used interchangeably with other image classification datasets in examples and benchmarks that use image classification models.

The LeNet-MNIST example used to show how to use this directly, but that now has a further abstraction in the form of a generalized TrainingLoop that takes in the dataset and handles the rest for you. If you step back to the example before that conversion, you can see how this works.

Again, sorry that this is more complicated than you'd expect, and isn't particularly well documented at present. We're hoping to improve upon that.

[Dane christian Neilson]

MT103/202 DIRECT WIRE TRANSFER
PAYPAL TRANSFER
CASHAPP TRANSFER
ZELLE TRANSFER
LOAN DEAL
TRANSFER WISE
WESTERN UNION TRANSFER
BITCOIN FLASHING
BANK ACCOUNT LOADING/FLASHING
IBAN TO IBAN TRANSFER
MONEYGRAM TRANSFER
IPIP/DTC
SLBC PROVIDER
CREDIT CARD TOP UP
DUMPS/ PINS
SEPA TRANSFER
WIRE TRANSFER
BITCOIN TOP UP
GLOBALPAY INC US
SKRILL USA
UNIONPAY RECEIVER

Thanks.


NOTE; ONLY SERIOUS / RELIABLE RECEIVERS CAN CONTACT.

DM ME ON WHATSAPP
+14234753476