Swift for TensorFlow v0.6.0 now available

[Brad Larson]

The v0.6.0 Swift for TensorFlow toolchain is now available, and Colab instances are now running this toolchain. The generated API documentation has been updated to reflect the changes in this version, and the release notes describe these API changes. Copying over from the RC1 announcement, these changes include:

• Changing the epsilon and momentum properties on BatchNorm and LayerNorm to be scalars.
• Removing a few deprecated comparison operators.
• Starting an internal restructuring involving the new Tensor module.

In addition, `Tensor.droppingOut` and `Tensor.transposed(withPermutations:)` have been deprecated and will be removed in a future release.

We've also added a new toolchain build for Ubuntu 18.04 that supports CUDA 10.1.

Few changes were made between RC1 and the official v0.6.0 release, with the macOS toolchain build remaining the same. The Ubuntu CPU and CUDA 10 toolchains should see a significant decrease in binary size between RC1 and the v0.6.0 release, but the functionality should remain the same.

As always, if you encounter issues with this toolchain or any of the nightly builds, feel free to file a GitHub issue on the relevant repository or to create a Jira ticket on bugs.swift.org.

[Rahul]

Hey Brad Larson,

I very much love S4TF project. But I find that sometimes reading documentation is difficult in comparison to that of PyTorch. For instance, maybe some newcomer to S4TF might not know/remember the formula for a given function, say `categoricalHingeLoss(predicted:expected:)`, then it becomes a difficult & lengthy process to Google it and then come back to continue learning and using it.

I believe it will be amazing if equations describing the functions (say, categoricalHingeLoss(predicted:expected:)), structures (say, Conv[x]D) ], and more alongside their usage examples are provided in documentation. Basically, please add three more things to S4TF documentation to make it easy to use:
1. Long description of ML concept. Link: https://pytorch.org/docs/stable/nn.html#transformer-layers
- For instance,
A transformer model. User is able to modify the attributes as needed. The architecture is based on the paper “Attention Is All You Need”. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 6000-6010. Users can build the BERT(https://arxiv.org/abs/1810.04805) model with corresponding parameters.
2. Equations describing machine learning concepts.
3. Example usage code for each concept.

Currently, the way documentation is designed sometimes hinders the process of using S4TF smoothly. Since Swift is a relatively new language for machine learning it needs to be cool from every viewpoint so as to attract new users. I think these important modifications (maybe more) to documentation will welcome a wide range of people into trying and using Swift for TensorFlow. Please take some action, it will be really helpful to me too!

- Rahul Bhalley

[Dan Zheng]

Hi Rahul,

One item on our agenda is categorizing Swift for TensorFlow API documentation (https://www.tensorflow.org/swift/api_docs) by semantic categories (e.g. layers, optimizers, loss functions, etc) rather than declaration kinds.

This should be done before the next release and makes it easier to find documentation for functions like `categoricalHingeLoss(predicted:expected:)`.

API additions and contributions are incremental, so detailed documentation may be a bit initially lacking, as you noted. Documentation improvements to tensorflow/swift-apis are certainly welcome!

Dan

--
To unsubscribe from this group and stop receiving emails from it, send an email to swift+un...@tensorflow.org.

[Brennan Saeta]

Hi Rahul et al.,

First of all, thank you so much for this feedback. I think you're absolutely right that we can (and should!) improve documentation. This is something I would like the core team to help improve, but we'd also very much welcome folks from the community to help us out here too! :-) I really like the template you described for how layers can be documented. Would it make sense to add something to CONTRIBUTING.md discussing documentation of public APIs (perhaps in addition to the code style section)? 

All the best,

-Brennan

[João Araújo]

Hi folks,

I can write all the equations for the losses, only need to know how to use LaTex within the documentation, do I have to use tags (like [latex][/latex]) or just $ ?

To unsubscribe from this group and stop receiving emails from it, send an email to sw...@tensorflow.org.

--
To unsubscribe from this group and stop receiving emails from it, send an email to sw...@tensorflow.org.

[Dan Zheng]

I can write all the equations for the losses, only need to know how to use LaTex within the documentation, do I have to use tags (like [latex][/latex]) or just $ ?

The API documentation generation tool (Jazzy) unfortunately doesn't support LaTeX - could you please write math in backticks (e.g. `x + x`) instead? It will appear with inline code style - example documentation for `Tensor.split(sizes:alongAxis:)`. Thank you!

 

Dan

[Rahul]

Looks good. If any help needed please let me know too. I’d also love to help in making the documentation better! 😃

[olzhas]

Thanks Brennan. Regarding your idea for CONTRIBUTING.md see one of the latest PRs as a start

To unsubscribe from this group and stop receiving emails from it, send an email to sw...@tensorflow.org.

--
To unsubscribe from this group and stop receiving emails from it, send an email to sw...@tensorflow.org.