Docs for SIG IO

[Edd Wilder-James]

Hi SIG IO!

As TensorFlow 2.0 approaches we want to ensure that projects like Addons and IO are linked in the right way from tensorflow.org so that users understand where to find important functionality. I was talking with Mark from our developer relations team about this and he mentioned that some degree of docs are going to be required for IO to get linked in.

Hence, I'm opening up this topic on the list and suggest it as a topic for next week's SIG meeting as well. I've included Billy, who directs TF's docs community, and Joana, who is now helping me with the SIGs here -- there could well be routes to recruiting from the community to help with this.

Mark, if you could share pointers perhaps to some examples of where this exists in Addons and could be done for IO, that would be helpful!

thanks

Edd

--

Edd Wilder-James, Open Source Strategy (ML & Data) at Google

Join us at TensorFlow World, Oct 28-31 

Github @ewilderj • @tensorflow • @kubeflow

[Yong Tang]

Hi Edd/Mark/Billy/Joana,

Thanks a lot for the help! I am sure with IO docs linked to tensorflow.org that will greatly help the community in finding very useful I/O related ops. We are happy to make any changes or related work that are necessary to get this done.

Thanks

Yong

---

From: 'Edd Wilder-James' via SIG IO <i...@tensorflow.org>
Sent: Friday, September 6, 2019 10:19 AM
To: SIG IO <i...@tensorflow.org>; Mark Daoust <markd...@google.com>; Joana Carrasqueira <joana...@google.com>; Billy Lamberta <bl...@google.com>
Subject: Docs for SIG IO

 

--
You received this message because you are subscribed to the Google Groups "SIG IO" group.
To unsubscribe from this group and stop receiving emails from it, send an email to io+unsu...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/io/CALW3V4fE13DSCvuQj9d8%3DejDmbgiNazTk3pBKK6UEgqsWTYckQ%40mail.gmail.com.

[Yong Tang]

Thanks Mark! I will play with the subsite template and see if we could get started with a page or two.

Thanks

Yong

---

From: Mark Daoust <markd...@google.com>
Sent: Friday, September 6, 2019 3:55 PM
To: Yong Tang <yong...@hotmail.com>
Cc: SIG IO <i...@tensorflow.org>; Joana Carrasqueira <joana...@google.com>; Billy Lamberta <bl...@google.com>; Edd Wilder-James <e...@google.com>
Subject: Re: Docs for SIG IO

 

Yes. 

I've added a links to tensorflow/io from the load and preprocess data section of the tutorials.

But I think just having some runnable example notebooks would be a good starting point, like tensorflow/addons for example.

https://github.com/tensorflow/addons/tree/master/examples

We'd need to workout some details, but this is the project template we've been using for every tensorflow subproject:

https://github.com/tensorflow/docs/tree/master/tools/templates/subsite

[Yong Tang]

Thanks Mark for the pointers. I played with addons' doc generator and it looks good. I may need to adjust a little bit for tensorflow/io's repo but I think will get there shortly.

Also one question about the docs, do we need to upload the generated doc directory `docs/api_docs/python/` to tensorflow/io's GitHub repo?

Thanks

Yong

---

From: Mark Daoust <markd...@google.com>
Sent: Wednesday, September 11, 2019 2:59 PM

To: Yong Tang <yong...@hotmail.com>
Cc: SIG IO <i...@tensorflow.org>; Joana Carrasqueira <joana...@google.com>; Billy Lamberta <bl...@google.com>; Edd Wilder-James <e...@google.com>
Subject: Re: Docs for SIG IO

 

Also please have a look at the tensorflow.org api reference generator.

Addons has a build docs script setup:

https://github.com/tensorflow/addons/blob/master/tools/docs/build_docs.py

If you copy that as a template we can put up an api reference section for IO as well.

LMK if you have any questions.

[Yong Tang]

Thanks Mark for the help! We have updated the repo and added the `tools/docs/build_docs.py` file to tensorflow/io. Now it is possible to generate the documentation markdown suite with:

```

1) Build whl file in wheelhouse/ directory

$ bash -x -e .travis/python.release.sh

2) Install tensorflow-io package:

$ python -m pip install wheelhouse/tensorflow_io-*-cp$(python -c 'import sys; print(str(sys.version_info[0])+str(sys.version_info[1]))')*.whl

3) Install tensorflow/docs pre-requisites:

$ python -m pip pip install -U git+https://github.com/tensorflow/docs

4) Generate Docs:

$ python tools/docs/build_docs.py

```

The docs are generated within docs/api_docs/python/.

Please note step 1) "Build whl file in wheelhouse/ directory" requires docker installed. If docker is not available on your build machine, please let me know so that I can update the script to make it compatible with the build machine setup on you side.

Also, step 1) will generate python 2.7, 3.5, 3.6, 3.7 wheel files. In step 2) I use command line to selectively install only the version that matches the `python` command line version.

Again, thanks a lot for the help, and let me know if there are any issues.

Thanks

Yong

---

From: Mark Daoust <markd...@google.com>
Sent: Thursday, September 12, 2019 10:34 AM

To: Yong Tang <yong...@hotmail.com>
Cc: SIG IO <i...@tensorflow.org>; Joana Carrasqueira <joana...@google.com>; Billy Lamberta <bl...@google.com>; Edd Wilder-James <e...@google.com>
Subject: Re: Docs for SIG IO

> do we need to upload the generated doc directory `docs/api_docs/python/` to tensorflow/io's GitHub repo?

No requirement. You can if you want.

We will generate that directory from the pip package as part of the site build/update pipeline.

[Mark Daoust]

Okay, 

Here are a couple of fixes to the api-reference-generation script: https://github.com/tensorflow/io/pull/487
I was just generating the docs from the default pip installation, which is what our site-building pipeline does.

 A few submodules fail to import. Anything that we can't get to import  won't end up in the site.

So the last thing, before we can publish on tensorflow.org, is that we need a copy of this template in tensorflow/io as a docs directory. Here is how tensorflow/addons implemented theirs.
It will at least need a landing page "_index.yaml". We can delete the "guides" and "tutorials" tabs until you have content to put there. 

Ideally the tensorflow docs team would be included as owners for that directory so it's easier for us to patch problems when they pop up. 

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/io/BYAPR10MB3736122B1207A2850AEB9FC8E28C0%40BYAPR10MB3736.namprd10.prod.outlook.com.

[Yong Tang]

Thanks Mark for the help! I have merged the PR. For the missing module you found, I have created an issue #489 to capture. Will work on those missing modules and get those fixed.

I also added tensorflow/docs-team with write access to the repo. Let me know if you encounter any issues.

We will looking into how to add tutorial and landing page shortly.

Thanks!

Yong

---

From: Mark Daoust <markd...@google.com>
Sent: Wednesday, September 18, 2019 10:02 AM

[Yong Tang]

Hi Mark,

We created an initial PR with page (https://github.com/tensorflow/io/pull/498). I mostly copied from Addons page, with book page removed initially. Wondering if there is a sandbox with page rendered we could look?

Thanks

Yong

---

From: Yong Tang <yong...@hotmail.com>
Sent: Wednesday, September 18, 2019 12:16 PM
To: Mark Daoust <markd...@google.com>

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/io/BYAPR10MB3736857B87C947283CEDFA1BE28E0%40BYAPR10MB3736.namprd10.prod.outlook.com.

[Mark Daoust]

> with book page removed initially.

Okay, so the _book.yaml file isn't a page, it's what tells the site how to layout the navigation buttons, table of contents, tabs and things like that.
So you will need the _book.yaml file, for this to work. 

>Wondering if there is a sandbox with page rendered we could look?

Once the _book.yaml file is added to the PR I can try staging it on the site.

I'll report back on the PR if I see any problems.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/io/BYAPR10MB373628D93EE2AE4610159679E2850%40BYAPR10MB3736.namprd10.prod.outlook.com.

[Yong Tang]

Thanks Mark! The PR has been updated. Let me know if there are any issue.

Thanks

Yong

---

From: Mark Daoust <markd...@google.com>
Sent: Monday, September 23, 2019 9:57 AM
To: Yong Tang <yong...@hotmail.com>
Cc: SIG IO <i...@tensorflow.org>; Joana Carrasqueira <joana...@google.com>; Billy Lamberta <bl...@google.com>; Edd Wilder-James <e...@google.com>; Ivelin Ivanov <ivelin.atan...@gmail.com>