Documenting private methods.

[Mattias Gyllsdorff]

Hello. 

Are there any plans to allow @doc to be used on private (defp) methods? I currently have several projects where we would like to generate one set of documentation for internal developers and one for external users/library users. 

Right now @doc on :defp, :defmacrop and :typep gets detected in module.ex, line 593 and turned into a error. 

[Gary Rennie]

Hi,

If you look in the Elixir source code, you will see that documentation for private functions simply uses comments:

https://github.com/elixir-lang/elixir/blob/8139ef0feec653ce573d45693898f5d863460e20/lib/elixir/lib/inspect.ex#L418

https://github.com/elixir-lang/elixir/blob/036e4e34a47900f70fabddcd7bfd26709272aae9/lib/eex/lib/eex/compiler.ex#L72

You can also find the same in Phoenix:

https://github.com/phoenixframework/phoenix/blob/5fc7eb829338dca3efd4a4d8f79f9e928d1bc34c/lib/phoenix/transports/long_poll.ex#L95

https://github.com/phoenixframework/phoenix/blob/698afa34805e4eb8e51b203a943f34da31afa84d/lib/phoenix/endpoint/instrument.ex#L85

Perhaps this would work for your internal documentation?

[Mattias Gyllsdorff]

Hello, thank you for your quick response.

I noticed that but I was hopping that there was someway to produce a external, stand alone representation of the internal documentations. I have gotten several request for a HTML view of the private functions from other internal devs who want to use it as a quick overview when they are looking at a unfamiliar module. 

[José Valim]

Elixir takes a pretty strong instance on public vs private and documentation vs code comments.

Private functions are meant to be implementation details. You can't even call them directly. Documentation are for folks that want to use your API without reading the source code. Code comments are for those *in the source code*.

So it doesn't make sense to document a private function because you can't ever call it. It is not your API. At this point, I would recommend you to ask yourself "what are you trying to achieve". Depending on the answer, you may do different things:

1. Use code comments (won't be exported)

2. Move the documentation you want to write for the private function to the public function that calls it (or the moduledoc)

3. Get the private functions and make them public so you can properly document and test them

And here is the point: even if we allowed you to document private functions because you have internal and external devs, how are you going to segregate this information in the generated docs? Would you build two sets of documentation?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/48409db0-0b20-4f47-91fd-910a239ccab7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mattias Gyllsdorff]

Private functions are meant to be implementation details. You can't even call them directly. Documentation are for folks that want to use your API without reading the source code. Code comments are for those *in the source code*.

 

I agree completely. The thing is, the ex_doc HTML generated from the @doc tags is often much more readable compared to unformatted code comments, especially for the other internal developers that haven't used the module before.

3. Get the private functions and make them public so you can properly document and test them

That is what I am currently doing since I want to use the doctests. But since the current documentation is for library users and not for the internal library developers I have to use the "@moduledoc false" command to hide it from ex_doc.

And here is the point: even if we allowed you to document private functions because you have internal and external devs, how are you going to segregate this information in the generated docs? Would you build two sets of documentation?

Yes, that is exactly what we would like to do. In our java projects we use the "-public" argument for the javadoc command and generate the integrator javadoc in one directory and the complete internal documentation in another directory. 

[Norbert Melzer]

I think this request is meant to create 2 sets of documentation. In a tool called doxygen which generates docs for various c style languages, you can do exactly this. By marking functions as internal, their documentation will be omitted by default, but you can pass a flag to generate docs with internal functions included.

Even if only available inside of the modulescope, it is very useful when you have private functions which are not only meant to separate code, but to reuse he code. This second set of documentation is a good help for people who want to participate in the project, while the first set of documentation is a good thing for people who want to use the project (Eg as a library).

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAGnRm4JqO5gew7Miq%3Dth1ZcHoOYJHthNkB9U_aM1fvefcJ4sSg%40mail.gmail.com.

[José Valim]

Yes, that is exactly what we would like to do. In our java projects we use the "-public" argument for the javadoc command and generate the integrator javadoc in one directory and the complete internal documentation in another directory. 

So I think this is the answer to your question. You don't want to document private functions, you want to be able to say some modules should be included in the documentation and sometimes they should not. :) 

[Norbert Melzer]

Sorry, Jose, its not about modules, it is about functions!

A module is accessible from the outside. You can't change that. You can discourage its use by documenting the fact that it is only meant for internal use, but you can't forbid it.

On the other side we do have modules which do have a public API which we can have documetation generated for. Thats fine. This is what a client should get when using our library. But while we code that module, we have functions f and g, which do both use the private function h, which we want to document also for internal use only. We want to let ex_doc run with a flag, that does give us a set of documentation where this function h is documented, since it is available to programmers working at (in contrast to "with") our library, such they can easily see from the generated documentation, that there is this function h which they can use while implementing a new public function foo.

I am working with a double screen setup and have my editor with the module I am working on at the left panel and a browser with the documentation open at the right pane. I am not using an editor which shows me an outline of available functions in a module, I do not have a splitpane with removed comments and function bodys. All I have is the left pane of the generated documentation as an overview of available functions. It would nice if I could see there, that there is already a private function available which does what I want before I do the work to reimplement it by a different name, which will be sorted out on Peer-Review.

I hope I was able to show that such a feature might be useful by describing my personal minimized working situation.

José Valim <jose....@plataformatec.com.br> schrieb am Mi., 9. März 2016 um 12:56 Uhr:

Yes, that is exactly what we would like to do. In our java projects we use the "-public" argument for the javadoc command and generate the integrator javadoc in one directory and the complete internal documentation in another directory. 

So I think this is the answer to your question. You don't want to document private functions, you want to be able to say some modules should be included in the documentation and sometimes they should not. :) 

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAGnRm4K20a7hFNTYMi1M9ScnH69J00xn_tkmqB6eEbY%2ByQtFiQ%40mail.gmail.com.

[Onorio Catenacci]

I agree with Jose on this issue.  If something is private it shouldn't be revealed via documentation.  Other developers using the module should only know about (and be able to rely on) publicly exposed functionality.  

I'm reminded of Steve Yegge's famous rant about API's and Amazon.  I can't find a direct link to the original blog post but here's an indirect one: https://plus.google.com/+RipRowan/posts/eVeouesvaVX

Here's the most relevant part for this conversation:

"His [Bezos' --Onorio] Big Mandate went something along these lines:

1) All teams will henceforth expose their data and functionality through service interfaces.

2) Teams must communicate with each other through these interfaces.

3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team's data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network."

Does that "no shared-memory model" sound familiar to anyone?  And tell me--who can you think of who can scale to the degree that Amazon can?  I mean whatever else Amazon may do wrong, they definitely do scalability right.

My point is that exposing internal details is a recipe for a non-scalable solution.  If a different team takes over maintenance of a code base then they can (and should) be examining internal code comments in great detail.  But exposing internal details is a quick recipe for pathological coupling and a hard to scale solution.  

If you really need that sort of internal documentation (and I suspect you really don't) create an internal wiki and let the various teams post their knowledge there.  It's not that hard to manage an internal knowledge base in that way.  But I really would avoid that because the more other teams know about internal details the more you promote a solution which will scale very poorly.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CA%2BbCVsvDP8CszyetCmFv8bE5hRV_%3Dp07wgjCwtcfOaW48mi_Ow%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Onorio Catenacci
 
http://onor.io

http://www.google.com/+OnorioCatenacci

[Norbert Melzer]

> I agree with Jose on this issue.  If something is private it shouldn't be revealed via documentation.  Other developers using the module should only know about (and be able to rely on) publicly exposed functionality.  

Again, we are talking about documentation for people who contribute to a library, not to people who are using it.

This means:

If I am a developer in the Foo-Team, I want to see what functions I do have available in the module I am currently working at, since I am able to use them and probably should to keep everything DRY!

If I am a developer who uses the Foo-library, all I need to see is the documentation for publicly exposed functions, since I weren't even able to use private ones.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAP%3DvNq_1rhikEE-G-6xYHWJ8PV9kt271ngb15%3DAgJYJeWm%3DsUg%40mail.gmail.com.

[Mattias Gyllsdorff]

I think we are all in agreement: module users should only look at the public API and never see the internal implementation. If the public API don't tell you what you need to know (is the algorithm O(log n), O(n), or O(n^2)?) then the documentation should be updated. It does not matter if the user is sitting in the next room and is working for the same company or if he/she is sitting on the other side of the world and working for our competitor. The @doc on the def is the public contract and it should contain everything the user need to know and should preferably never be broken.

The internal module developer on the other hand needs access to the internal implementation documentation so he/she can see what the function is supposed to do and what parameters it can handle. In my experience a external wiki always suffers from bit rot and decays until no one uses it. The code comment next to the function is generally the only information that gets updated. 

The @doc tag on a defp/protected method defines a contract for the module developer in the same way the @doc on a def/public method defines a contract for module users.

I actually once had a project where the devs only had access to javadoc with the public classes and functions. Access to the documentation of the protected and private functions was granted on a case by case basis and required a reason.

[Mattias Gyllsdorff]

Well actually...

I already have a script that removes @doc tags and adds "@moduledoc false" to certain modules depending on who the documentation is targeted to. The problem is that there is no way to get ex_doc to generate documentation for the defp methods. :P 

[Onorio Catenacci]

So ok, I follow what both of you are saying but why do you want an HTML version of internal documentation?  Can't your developers read comments?  I think you're asking for a change to a fairly fundamental feature of the language for one corner use case.

--

Onorio

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CA%2BbCVsthpZzo7tP%3DZqD%2B1hkxsBMsJcqz2FAhnctFBmaeMhSs9A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Norbert Melzer]

As I already said, if the private stuff is alongside the public stuff in the generated docs, it is easier to find :) There are documentation blocks multiple screenpages log, while functions are very short and hidden inbetween.

Personally I do not really need that feature, from some work I had to do for my university I have been in an even worse situation (documentation wise), there was nearly no documentation of the API.

All I try to do here, is to help the OP to make its position more clear, since I have the feeling that I was the only one who understood what he wanted from the very beginning.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAP%3DvNq_tFYY4_LRgzat0PifqAMuq8vs2XoChULEADLquxjYobg%40mail.gmail.com.

[Onorio Catenacci]

Ok, Norbert, I think I was misunderstanding you. Thank you for clarifying. 

That said, I still think that Mathias is asking for a change to the language to satisfy a use case that is only a concern for him. If anyone else needs that functionality, they should speak up.

--

Onorio

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CA%2BbCVsvU-dggD3oqVN9Hx%2BBneB-inRF_9DXxCBR2E2FnBb-fdQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Mattias Gyllsdorff]

Oh, I was mostly wondering if it was a intentional feature or just a implementation detail, since it does not seems to be documented anywhere. :) 

Another reason I asked this question was that one of my colleagues tried to use the h() command on a private method and asked me why it did not work.

I have some spare time coming up so if this is something that José and the other maintainers want to change I can take a look and see if I can get something working and send a PR for module.ex and one a PR to ex_doc. 

[Brad O'Hearne]

Mattias, 

Whether public or private methods/functions/modules/whatever, and regardless of language, documentation is valuable, and a good idea, in all cases. There's no plausible argument why having less information describing code function and design intent is a good idea, simply because of visibility declaration. Because a function is public or private doesn't in any way indicate a greater or lesser need to document it -- it may indicate a greater need for a particular audience to read it, but it does not diminish any need to document. Documentation = good, whether public, private, or inline, whether a development team of 20 or more is working on the code, or whether there's only one person working on the code. 

Presentation and the intended audience of documentation is a different issue. The 2 parties being generally discussed here, if an external API is involved: 

1. The API consumer. This audience should have documentation of just the public elements they should be aware of. Nothing private should be presented in their doc. Implementation details which affect the behavior of the API should be documented for them.
2. Developer. This audience should have documentation of all elements, public and private.

There isn't any reason why either party above has more or less need than the other for clean, readable, traversable documentation, separate from source code -- and that has nothing to do with the fact that source code is available (and especially so, because source code may also be inconsistent or wrong, depending on the intent communicated in the doc). Because a function is private doesn't change the usefulness or desirability of having readable HTML doc generated. Additionally, the term "API" is kind of misused anyway -- while it seems that we typically use that for public functions that third parties will consume, the truth is that every function is an API, whether public or private, they just have different consumers. Private functions are just as much an interface as a public function -- it defines a way which a specific set of functionality is facilitated and addressed -- the difference is in its potential callers...still an interface though. 

FWIW, we have the same need on several projects too -- to generate documentation of public / private function doc, apart from source code, in a format that can be viewed in a browser, easy to read, and quickly traversable. This is useful to developers working on the code, and newcomers trying to understand how a codebase is organized. Being able to choose to how doc is generated, whether it includes private functions or not, would be a great improvement we would use. 

Yours is a good post and good request.

Brad

Because a public method may be While a public method may grab the eyes of an audience not familiar with the internal implementation, and therefore 

[José Valim]

the truth is that every function is an API, whether public or private, they just have different consumers. Private functions are just as much an interface as a public function -- it defines a way which a specific set of functionality is facilitated and addressed -- the difference is in its potential callers...still an interface though. 

I completely disagree with this statement. An interface is "a point where two systems, subjects, organizations, etc. meet and interact". In Elixir this maps to modules and it doesn't make sense to talk about interfaces for private functions because they can't be accessed anywhere outside of its own module.

If you feel you need to document private code, make its visibility public, possibly moving to another module. I would gladly review a proposal where you can control which modules are part of the documentation (so you can split internal and non-internal ones). But private functions are not an interface and allowing private functions to be "documented" is not only misleading, because you can't invoke them outside of the module anyway, but also a bad practice because you can't even doctest (or test) the function you are documenting.

[James Gray]

On Wed, Mar 9, 2016 at 11:58 AM, Brad O'Hearne <brad.o...@gmail.com> wrote:

There's no plausible argument why having less information describing code function and design intent is a good idea, simply because of visibility declaration.

I'm pretty sure I can think of several plausible arguments.  Here's one.

I feel it's pretty well known that more information isn't always superior in other fields and life in general.  Just to give one example, we have good evidence that cancer screenings done too often lead to an increase in unnecessary procedures being performed on patients.

I see no reason that this wouldn't apply to programming as well.  In the case of private functions being documented, it could lead you to rely on code slated for removal or who knows what else.

I can think of compelling reasons not to document 100% of the public API.

James Edward Gray II

[Brad O'Hearne]

On Wednesday, March 9, 2016 at 11:33:24 AM UTC-7, José Valim wrote:

 An interface is "a point where two systems, subjects, organizations, etc. meet and interact". In Elixir ...

Jose -- I'm not sure your definition above is actually much different than what I said -- a place where two entities meet and interact. Maybe using the word "interface" causes the discussion to stray on semantics, but the fundamental nature of the relationship -- that one entity is providing functionality to the other through a contract -- is the same whether public or private. But to the point of the post (documentation), because something is a private function which facilitates implementation details vs. a publicly exposed function intended for external API, that doesn't change the need for documentation at all -- it just changes the audience the doc is appropriate for. 

When you buy a car, they put an owner's manual in the glove box, which tells you about everything on the dashboard and how to operate the vehicle. That is documentation for the consumers of the external API -- drivers. But drivers aren't the only audience needing doc, nor is the only purpose of doc to drive the car -- some of the doc is for servicing the car and knowing how it works under the hood.  Mechanics / service centers, manufacturers / vendors parts, and the manufacturers own engineers, operations folks, assembly technicians, etc.. They all need their own sets of technical doc, which the manufacturer creates and publishes. The manufacturer doesn't just publish an owner's manual for the driver and tell everyone else "can't you pop the hood, take the engine apart, and figure it out yourself?"

Anyway, I'm happy to agree to disagree in regards to documentation. I haven't ever seen a codebase improved by "less documentation", or someone become a better developer by "documenting less". My real purpose was to give props to Mattias, after the thread which ensued from his original post, I wanted him to know that another organization (which values thorough documentation not only for consumers, but its own internal developers) has the exact same need.

[Onorio Catenacci]

valuable, and a good idea, in all cases. There's no plausible argument why having less information describing code function and design intent is a good idea, simply because of visibility declaration. Because a function is public or private 

There's a very simple, very plausible reason for this.  There are only 24 hours in the day. Time taken preparing documentation is time that can't be used for other things like testing and debugging.  Even if someone else is taking care of the testing and the debugging, if they can't touch the code till we're done with it, they're blocked. All software development activity is a tradeoff between competing demands on our time.  It's not as if we have a magic wand that can allow us to work on multiple activities at once. 

Glenn Myers in "The Art of Software Testing" made the very salient observation that if we had unlimited time for testing and unlimited resources we might be able to catch all bugs. The problem is that we simply don't have unlimited time or unlimited resources.  So we have to try to figure out the strategy that will catch most bugs given the limits of our time and resources. 

--

Onorio

 

[José Valim]

>  that one entity is providing functionality to the other through a contract -- is the same whether public or private. 

You can't remove "interface" from your description and replace it by "contract". :) Contracts and interfaces do not apply to private functions. That's the purpose of private functions in the first place, nothing outside of the current module can depend on it, there is no contract, no interface.

As a matter of fact, unless you are reading the source code, *they do not exist*. The compiler may as well shuffle the private function names into hashes or inline them for performance. It makes no sense to access any information about private functions from outside of the source file, be it from a HTML page or your terminal, because for all purposes they were never there.

 

When you buy a car, they put an owner's manual in the glove box, which tells you about everything on the dashboard and how to operate the vehicle. That is documentation for the consumers of the external API -- drivers. But drivers aren't the only audience needing doc, nor is the only purpose of doc to drive the car -- some of the doc is for servicing the car and knowing how it works under the hood.  Mechanics / service centers, manufacturers / vendors parts, and the manufacturers own engineers, operations folks, assembly technicians, etc.. They all need their own sets of technical doc, which the manufacturer creates and publishes. The manufacturer doesn't just publish an owner's manual for the driver and tell everyone else "can't you pop the hood, take the engine apart, and figure it out yourself?"

Anyway, I'm happy to agree to disagree in regards to documentation. I haven't ever seen a codebase improved by "less documentation", or someone become a better developer by "documenting less". My real purpose was to give props to Mattias, after the thread which ensued from his original post, I wanted him to know that another organization (which values thorough documentation not only for consumers, but its own internal developers) has the exact same need.

I am not disagreeing documentation may have different audiences. I am not asking to document less. I have already said I would love a proposal on making it easier to generate documentation with (or without) extra modules. My only point is that private functions are completely internal and, as such, they should not be documented. If you want to document a private function, move it to separate module that won't be part of your "user-facing manual" but only part of "service center guide".

[Brad O'Hearne]

On Wednesday, March 9, 2016 at 1:18:20 PM UTC-7, José Valim wrote:

Contracts and interfaces do not apply to private functions.

Isn't every function signature in effect a contract in its scope of visibility? I think this point is suffering due to semantics. Forget the words "contract" and "interface" and instead replace with "how it is called and what it does". 

 

 My only point is that private functions are completely internal and, as such, they should not be documented.

If your statement above would have been followed by "...for external API users" I could follow along, but as a blanket statement about documentation, I don't agree. Because a function is private, that in no way affects the need to capture useful information about how it works, or its design intent. Do you think that once under the hood of the Windows API, OS X API, IOS API, Android API, that those codebases aren't documented? The majority of the documentation in those cases are private. Additionally, to the first point above, you'll commonly hear discussion of "public API" vs. "private API", meaning that something isn't an API only if it is public. Private is *also* considered an API, it just has a different consumer audience. 

If you want to document a private function, move it to separate module 

If in order to document something you have to re-architect your code, I think that pretty strongly suggests the category of "you're doing it wrong". Any private or public function in any module should be able to be documented.  

You don't have to reply if you don't want to, Jose. I'm probably a lost cause for the "don't document" crowd...I'm firmly entrenched in the opposite camp, not just ideologically, but from a career of dealing with undocumented codebases and OSS projects that greatly diminished their pragmatic utility because they viewed documentation as something separate from development, that they didn't have time for. I'm fine to agree to disagree. But I am extremely curious about what kind of software projects and organizations benefit from less documentation (rhetorical...no need to reply :-).

 

[matteo brancaleoni]

Just throwing my  2cents here...

while I understand the idea behind not documenting (except with code comments) private functions,

in my daily work happens that I need to have some sort of docs about private functions for these reason:

* remember what I/others have done :) without having to dig into all source files

* if in some module I need to do something that maybe I (or someone else) have already done in a private function,

  having a quick doc reference is helpful because helps a lot in finding that private fun and slap it into a public module,

  in order to be able to reuse it and avoid code duplication.

digging into source code to read comments is something you cannot do everytime if the codebase is big and

handled by more than one person...

Maybe adding something @docp like defp in order to split documentation between private and public function.

again, just my 2c

[José Valim]

Private is *also* considered an API, it just has a different consumer audience. 

Again, I am not disagreeing with this. When someone says "Private API", they are talking about API intent and not function visibility. In fact, even in languages where you can document private functions, I would expect the biggest part of the Private API (intent) to still be made of public functions (visibility).

 But I am extremely curious about what kind of software projects and organizations benefit from less documentation (rhetorical...no need to reply :-).

You are once again implying I am advocating for less or no documentation. Twice in the same paragraph. This is extremely frustrating because that's exactly the opposite I have been arguing and advocating all along: if you want to document it then it deserves the same treatment as any other documented code.  It is fine to disagree but don't distort my words.

[Brad O'Hearne]

On Wednesday, March 9, 2016 at 2:51:40 PM UTC-7, José Valim wrote:

You are once again implying I am advocating for less or no documentation...It is fine to disagree but don't distort my words.

Here is what you said: 

> My only point is that private functions are completely internal and, as such, they should not be documented. 

and then you also said this: 

> But private functions are not an interface and allowing private functions to be "documented" is not only misleading, because you can't invoke them outside of the module anyway, but also a bad practice because you can't even doctest (or test) the function you are documenting.

The original poster requested a way to include his private function documentation in generated doc, and you have answered that his private function documentation should not have been created in the first place, and as a rule, doing so is bad practice. How are you not advocating for less documentation here?

[Kyle Hanson]

I advocate for more documentation, but I personally do not see the point in having documentation generated separate from the source code. If you can't find a function in a module, you might want to consider refactoring. Your text editor should allow you to collapse all functions. And since you can only use it in the module, you already have that module open in your editor. 

Even if its not valuable to me, I can understand why some might want it. Its hard to imagine restricting someone because they want to document more, but I suppose sometimes restrictions can help make code "disciplined". 

As an interim solution, one possibility (obviously untested) is to generate the normal docs, then replace all defp with def using sed (since elixir throws a warning on doc strings attached to private functions you might also have to comment/uncomment those too). Then you can just revert those changes after you generated your private docs. A bit of a cumbersome solution, but one that lets you get started seeing if they are useful.

--

Kyle

--

You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/6b85df3a-2840-4a85-bea0-e30c2f189110%40googlegroups.com.

[José Valim]

Sigh. What about including the whole context?

> I am not disagreeing documentation may have different audiences. I am not asking to document less. I have already said I would love a proposal on making it easier to generate documentation with (or without) extra modules. My only point is that private functions are completely internal and, as such, they should not be documented. If you want to document a private function, move it to separate module that won't be part of your "user-facing manual" but only part of "service center guide".

> If you need to document private code, make its visibility public, possibly moving to another module. I would gladly review a proposal where you can control which modules are part of the documentation (so you can split internal and non-internal ones). But private functions are not an interface and allowing private functions to be "documented" is not only misleading, because you can't invoke them outside of the module anyway, but also a bad practice because you can't even doctest (or test) the function you are documenting.

Emphasis (bold) mine. Italic is the snippet you decided to "quote".

Not only I make it quite explicit I am not asking to write less documentation, I also propose alternatives that would work today. I am not saying: "don't write documentation". I am saying: "make the private function you are documenting public alongside the documentation you wrote". In both text snippets I also explicitly welcome proposals to generate documentation targeted to different users.

[José Valim]

To be a 100% clear and avoid any further misinterpretation and misquoting of my emails. I want to requote my first reply in this thread.

You can't document private functions. You can however:

1. Use code comments (won't be exported)

2. Move the documentation you want to write for the private function to the public function that calls it (or the moduledoc)

3. Get the private functions and make them public (in the same or another module) so you can properly document and test them

"write less documentation" was never listed as an option.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Onorio Catenacci]

So this is what the person who posted the original question (Mattias) said later on:

"Oh, I was mostly wondering if it was a intentional feature or just a implementation detail, since it does not seems to be documented anywhere. :) 

"Another reason I asked this question was that one of my colleagues tried to use the h() command on a private method and asked me why it did not work."

1.) Yes, this is an intentional feature. This actually is documented here: http://elixir-lang.org/docs/stable/elixir/writing-documentation.html under the heading "Privacy".  I think you're partially right though--it's not spelled out in black and white--more implied by the text. Note this passage a bit further down especially:

Elixir makes the difference between documentation and code comments. Documentation are for users of your API, be it your co-worker or your future self. Modules and functions must always be documented if they are part of your application public interface (API).

Code comments are for developers reading the code. They are useful to mark improvements, leave notes for developers reading the code (for example, you decided to not call a function due to a bug in a library) and so forth.

In other words, documentation is required, code comments are optional.

2.) h() doesn't work on private functions because they are _private_.  This, again, is intentional--not an error or a mistake. BTW not to split semantic hairs (which there seems to be plenty of in this conversation thread) but it's not correct to refer to them as "private methods".  They are private functions--because they're defined in modules and while modules may bear a superficial resemblance to classes they're not the same thing.  We need to remember that we're not dealing with OO any longer when we're dealing with Elixir. 

I'm taking the time to answer this because I don't think anyone ever really did address your clarification Mattias. 

--

Onorio

[José Valim]

Thanks for the analysis Onorio. I will improve those docs tomorrow for further clarity.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAP%3DvNq_TG7X2GaSRHQMX2pezapvcsMAMUhU-ic%2Bp%2BP4H60UXRw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Onorio Catenacci]

You're welcome Jose.  I know you're a busy fellow; I can make a PR if you'd rather.

--

Onorio

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAGnRm4KvHe302qjv6C6hNv8mfRdZer%3DV3TKym_w0yJa07PE32Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Bradley O'Hearne]

I appreciate any attempt to suggest alternatives. And in the equal spirit of suggestion and cooperation, here’s why these suggestions don’t satisfy the need: 

 

1. Use code comments (won't be exported)

Inline comments / having to read source code != exportable, easy to read, traversable documentation on module and function declarations / signatures. 

2. Move the documentation you want to write for the private function to the public function that calls it (or the moduledoc)

This has a number of problems. First, the documentation belongs with what is being documented, the private function, not in its callers — restated: callers do not define a function’s purpose, the function does. Second, if the documentation is moved into the calling function itself, that again is inline code, not exportable documentation. Third, if the documentation is moved into a public calling function’s exportable documentation, that’s propagating implementation details into the public API for a different audience than intended for the private function. Finally, a private function may be called by multiple public functions, not just one — you’d have to include the documentation for the private function in all of them to have the documentation available where used. 

3. Get the private functions and make them public (in the same or another module) so you can properly document and test them

This isn’t documenting a private function. It is changing your code architecture to accomplish a documentation effort. It also has a bigger consequence than documentation — it is publicly exposing functionality not intended to be visible to external callers.

The best summary statement for the whole situation is this: 

You can't document private functions.

In short, it would be beneficial to be able to document private functions. And that is why I originally posted — to echo the need. In our case, we have zero need for any public API doc to any external parties. We have 100% need for internal code doc for our developers, including private functions. Yes, we can get a certain amount of mileage from public function doc, but it doesn’t provide all of what we could have if private functions could be documented. With developers as a target of this doc, we are not looking to stop presentation of our doc at the outside of the black-box, leaving its implementation details unknown; we want to document how it works internally as well…meaning private functions too. 

This is a good area for improvement. 

B

[Josh Adams]

I just wanted to make one point, inline below.

>> 3. Get the private functions and make them public (in the same or another module) so you can properly document and test them

> This isn’t documenting a private function. It is changing your code architecture to accomplish a documentation effort.

I think this is the crux of the miscommunication. It truly isn't changing your architecture. Outside of the compile time source code of the module in question, they truly do not exist. Literally no one can use them unless they are presently editing that source code.

For this reason, I do not find any reasonable argument that they should be documentable. Failing to document them outside of the context of editing the source file is the single most architecturally true thing to do, if you appreciate what they are in the context of the beam.

There's my $0.02 in a thread that took a tone I was disappointed to read.

Josh

[John W Higgins]

https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/module.ex#L593-L598

Go modify this to your needs - compile a custom version for doc building internally - and ignore the warnings on the standard version.

All problems solved.

John

P.S. Bonus points awarded for inverting the code and having it print only private docs.

[Onorio Catenacci]

Hi José,

I submitted a PR to better document the behavior of @doc with relation to private functions.  Just so you know you don't need to worry about it.

--

Onorio

--

You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAPhAwGy419R2KF6Es_59uth2oagF-%3DBZHuhnCnRSkaFO%3DQ%3DHGw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Ben Wilson]

If you want to know how a given elixir project works, you're gonna start at the high level. What processes does it start, what modules does it have, what do each of them do at a high level.

Your next question might be "how do other parts of the system interact with this particular module". Public function documentation completely 100% cover this scenario because it's impossible for other modules to call anything else.

Now you want to know "how does this module work internally". There is no substitute for reading the code here. Suppose all functions even private ones were documented. You still don't know how it works. The canonical source for HOW a function works is the function code itself, there is no other.

In the end, I think the real difference here is a difference in philosophy about the purpose of Documentation and its relationship to the code it documents. Elixir's stance as best I can tell is that documentation's job first and foremost is to describe how to USE a module. If you believe that documentation should exist to enumerate the precise internal mechanics of a module then that's a fine stance to have, but it isn't one shared by the core team as best as I can tell. Nothing does that better than the code in conjunction with inline comments explaining reasons for particular choices.

[Brad O'Hearne]

On Wednesday, March 9, 2016 at 11:12:59 PM UTC-7, Josh Adams wrote:

There's my $0.02 in a thread that took a tone I was disappointed to read.

Josh....thanks for the reply, and I'm with you. It has always been a friendly discussion to me -- I entered this thread for one purpose: to encourage the original poster, and echo his need. Optimism views need as opportunity, and there was an opportunity here to respect and meet a legit need. I wish that was how it was viewed, rather than brushing off the need by declaring private function doc as bad programming practice, or a waste of time. The resolution could have been an improvement which provided an option which satisfied both approaches. But...I'll always be full of optimism. 

[Paul Schoenfelder]

 I wish that was how it was viewed, rather than brushing off the need by declaring private function doc as bad programming practice, or a waste of time

You're either intentionally misconstruing José's argument or you aren't listening to what he's saying, but he's neither brushing off the request, nor calling it a waste of time. He's explained this more than once already, so if you're trying to be friendly, it does not come across that way. Insisting he (and others of us) are saying something we are not is hardly the way to make meaningful progress.

Nobody would argue that private functions should not be documented, or that it's a waste of time, but that documentation *only* makes sense in the context of working in a given module, i.e. code comments. I regularly change private functions, throw them away, rename them, etc - documentation for them is minimal, because the functions themselves should be small and focused, easily understood with a moment's review, documenting them with comments is usually only to leave a trail of reasoning for something that may seem strange, or maybe is a temporary hack, or maybe an algorithm that came from StackOverflow or something. Personally, I mark private functions with typespecs, so that I can jump to the definition, see the spec, and know exactly what to expect. Writing full docs for them like I would a public function just doesn't make sense to me in this context. It does make sense to me if your modules are huge, with many private functions, or if your private functions are large and complex - but the solution for that shouldn't be documentation - it should be breaking up the modules, or breaking up those large functions into smaller ones.

I'm sure there is a situation where you would want such docs, but I don't think it's a use case that needs to be covered by Elixir. You could relatively easily implement this solution for your apps, by forking ex_doc, adding a macro that accumulates @docp attributes in a module, and then adding docs from that attribute to the set that ex_doc loads when it generates the HTML documentation. Such a fork would be pretty easy to keep in sync with upstream. Maybe it's not the solution you, or others with this need, would like, but it is a solution worth considering.

Paul

[Rich Morin]

On Mar 10, 2016, at 07:04, Ben Wilson <benwil...@gmail.com> wrote:
> In the end, I think the real difference here is a difference in
> philosophy about the purpose of Documentation and its relationship
> to the code it documents. Elixir's stance as best I can tell is
> that documentation's job first and foremost is to describe how to
> USE a module. If you believe that documentation should exist to
> enumerate the precise internal mechanics of a module then that's
> a fine stance to have, but it isn't one shared by the core team
> as best as I can tell. Nothing does that better than the code in
> conjunction with inline comments explaining reasons for particular
> choices.

Elixir isn't exactly "functions all the way down"; it has actors at
the top and macros at the bottom. In the middle, it uses modules
for namespace management. These are all key structuring mechanisms.

Knowing what the current level of granularity (eg, actor, module,
function, macro) claims to do is important to anyone working on the
code. Header comments should provide this information, along with
appropriate hints about data structures, algorithms, etc. The code
and inline comments then provide information on how the structuring
mechanisms are connected and used.

Elixir's mapping of code and comments into online documentation is
very pleasant and useful. I would like to see it work for every
level of interest and use case, without arbitrary restrictions such
as "we don't allow online documentation of this item".

If I want to look at some documentation online, I should be able to
do so. Make it an option, print out a warning message; whatever.
Just don't turn arbitrary restrictions into features:

feature: bug
option: bug with a workaround

-r

--
http://www.cfcl.com/rdm Rich Morin r...@cfcl.com
http://www.cfcl.com/rdm/resume San Bruno, CA, USA +1 650-873-7841

Software system design, development, and documentation

[José Valim]

Rich, I am assuming that when you say "arbitrary restriction", you are referring to the fact private functions can't be documented. If that is the case, why are we calling them arbitrary restrictions if the rationale behind the decision was explained over and over again?

I am not going to change the current behavior. There is a reasoning behind it, regardless if someone agrees with it or not. If you really want this feature, roll your own. The language is extensible for a reason.

PS: As I said in another email, I would still gladly consider proposals that make it easy to generate different docs from different modules, targetting different audiences. If interested, please open up a new topic. This is my last reply on this particular thread.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/A00F4070-71DB-4BFC-9D92-8C2497979A56%40cfcl.com.

For more options, visit https://groups.google.com/d/optout.

--

[Brad O'Hearne]

On Thursday, March 10, 2016 at 11:31:08 AM UTC-7, Paul Schoenfelder wrote:

Nobody would argue that private functions should not be documented, or that it's a waste of time...

That's exactly what's been argued:

> Writing full docs for them like I would a public function just doesn't make sense to me in this context.

> My only point is that private functions are completely internal and, as such, they should not be documented. 

> There are only 24 hours in the day. Time taken preparing documentation is time that can't be used for other things like testing and debugging.

> I'm sure there is a situation where you would want such docs, but I don't think it's a use case that needs to be covered by Elixir.  You could relatively easily implement this solution for your apps, by forking...

> I am not going to change the current behavior. 

Argue with those, not with me. Those statements are straightforward, and the full context in which they appeared is in no way misrepresented by those quotes above -- in fact, they are further developed and supported within their full context, so there's no need to cut/paste a paragraph for each, which no one would read anyway -- read the thread, they are all there, and they say exactly what they say. I just think it is a shame that when an opportunity to provide an optional solution, which in no way disrupts current usage, and apparently can be "easily implement[ed]", the door gets slammed for reasons other than technical ones. 

No one needs feel slighted that not everyone agrees with them, or that the challenges before other developers present use cases that might not have been considered by you before (which is clearly the case, not just with this issue, but others). Raising these ought to be appreciated, not ostracized, because they broaden the perspective on problems out there and can help improve the versatility of the language. Taking a simple thing like a request to improve the utility of documentation, and using it as a means to try to bludgeon someone else as a practitioner of bad programming practice is unnecessary. A little less shoot-the-messenger would be welcome, and might just lead to some helpful new perspectives and additions to Elixir, which I think is why we are all here in the first place.

Onward. 

[Brad O'Hearne]

Rich -- You framed the issue and need perfectly, and are 100% right. I wish I would have known your post was coming...I would have withheld all comment and deferred to you. Unfortunately it has become a "shall not be named" issue (seems that Elixir has a collection of those), but it matters to me to hear another developer who understands the need. 

I appreciate you posting. 

[Bernd Meyer]

I'm getting a little concerned following this thread. Perhaps it's just a misunderstanding.

I understand and agree with the reasons not to support the documentation of private functions in the ExDoc. There is a tradeoff between specific use cases and the objective not to confuse all other users about what private means.

However I do not understand why this use case should be dismissed. There are certainly many customs, work flows etc. in the industry many people might not approve. But should the people working in these environment not be allowed to use Elixir?

IMHO the solution should be solved through a custom package on hex.pm. developed by interested parties. If there are obstacles to implement this custom package I hope their questions will be welcome.

Keep up the good work!
Thanks, Bernd

[Ben Wilson]

"Unfortunately it has become a "shall not be named" issue (seems that Elixir has a collection of those)". This. It's this kind of stuff that has been a source of frustration in this thread. Languages and language designers take stances on certain issues, because decisions need to be made. Jose said that his decision is to not implement this feature. That is not the same as saying it is an issue that "shall not be named".

[paul.r...@gmail.com]

Because this thread went for too long, some developers may be worried about contributing to future discussions, so I want to share my experiences and hopefully end this on a positive note.

The Elixir community has been really welcoming since day 1. Folks are glad to review code, questions are answered quickly and discussions are often quite productive. I am always impressed we have the language designer (Jose) to participate in those discussions and share the rationale behind many of those decisions, not only here but also on IRC, Stack Overflow, Twitter, etc.

That said, it is a little disheartening to see his efforts being taken out of context. His replies explained how documentation and private functions work in Elixir, which was insightful, and he also gave alternatives for the poster to tackle his problem. While I am also personally inclined to document private functions, I appreciate an explicit statement the language won't change so I can move on and solve this problem in my own project whenever it arises.

I have never been told a question should not have been asked (quite the opposite). I have never stumbled upon issues that "shall not be named" (and I don't think this thread is one of them either). Everyone I have met shares quite similar experiences.

Thank you Jose and everyone that has helped me so far!

[Rich Morin]

On Mar 10, 2016, at 15:57, José Valim <jose....@plataformatec.com.br> wrote:
> Rich, I am assuming that when you say "arbitrary restriction", you
> are referring to the fact private functions can't be documented. If
> that is the case, why are we calling them arbitrary restrictions if
> the rationale behind the decision was explained over and over again?

I'm sorry if I caused offense; I was afraid that "arbitrary" might be
a problematic word choice. FWIW, I did NOT mean "based on random
choice or personal whim, rather than any reason or system". I meant
something like "unrestrained and autocratic in the use of authority"
(and I was not accusing you of this, either).

However, one of the reasons I gave up on golang was the fact that it
makes a lot of decisions that are "unrestrained and autocratic in the
use of authority". For example, unused variables or functions are
treated as fatal errors. So, I'm a bit sensitive on this topic.

> The language is extensible for a reason.

Indeed. Perhaps someone with a strong use case for this feature
will create a modified version that supports online documentation
of private methods, etc. If they do so, I would suggest that the
generated docs clearly distinguish the method's status.