Writing to a JSON file

[paul.wilding]

I'm trying to write a json serialization of data to a json file stored in my plugin directory. The plugin is called buildrequests and the file is found at buildrequestsplugin\buildrequestsplugin\buildrequests\htdocs\json

 

The error message written to the log is:

Trac[buildrequests] ERROR: Exception occured writing to file: [Errno 2] No such file or directory: 'buildrequests/json/breqs.json'
Traceback (most recent call last):
  File "build\bdist.win32\egg\buildrequests\buildrequests.py", line 109, in process_request
    file = open(breq_file, 'w')
IOError: [Errno 2] No such file or directory: 'buildrequests/json/breqs.json'

where breq_file is 'buildrequests/json/breqs.json'.

The same occurs if the breq_file is changed to 'buildrequests/htdocs/breqs.json'. 'buildrequests' is defined as the directory to reference to my htdocs and works for my javascript and css files and I have a reference in package_data in setup.py to 'htdocs/json/*' and so I'm running out of ideas - the only thing I can think of is that it might work if I were to move the file outside the main plugin directory i.e. outside the egg?

[Steffen Hoffmann]

Am 17.07.2012 18:45, wrote paul.wilding:
> I'm trying to write a json serialization of data to a json file
> stored in my plugin directory. The plugin is called buildrequests and
> the file is found at
> buildrequestsplugin\buildrequestsplugin\buildrequests\htdocs\json

As a side-note I think it's poor design to write into htdocs, since this
is meant to be served by web servers as static, cache-able content.

> The error message written to the log is:
>
> Trac[buildrequests] ERROR: Exception occured writing to file: [Errno
> 2] No such file or directory: 'buildrequests/json/breqs.json'
> Traceback (most recent call last): File
> "build\bdist.win32\egg\buildrequests\buildrequests.py", line 109, in
> process_request file = open(breq_file, 'w') IOError: [Errno 2] No
> such file or directory: 'buildrequests/json/breqs.json'
>
>
> where breq_file is 'buildrequests/json/breqs.json'.
>
> The same occurs if the breq_file is changed to
> 'buildrequests/htdocs/breqs.json'. 'buildrequests' is defined as the
> directory to reference to my htdocs and works for my javascript and
> css files and I have a reference in package_data in setup.py to
> 'htdocs/json/*' and so I'm running out of ideas - the only thing I
> can think of is that it might work if I were to move the file outside
> the main plugin directory i.e. outside the egg?

Did you implement both methods of the ITemplateProvider? This is
required to reference files from the egg within you Trac plugin code.

Yours,

Steffen Hoffmann

[1]
http://trac.edgewall.org/wiki/TracDev/PluginDevelopment/ExtensionPoints/trac.web.chrome.ITemplateProvider

[Olemis Lang]

On 7/17/12, Steffen Hoffmann <hof...@web.de> wrote:
> Am 17.07.2012 18:45, wrote paul.wilding:
>> I'm trying to write a json serialization of data to a json file
>> stored in my plugin directory. The plugin is called buildrequests and
>> the file is found at
>> buildrequestsplugin\buildrequestsplugin\buildrequests\htdocs\json
>
> As a side-note I think it's poor design to write into htdocs, since this
> is meant to be served by web servers as static, cache-able content.
>

<OT?>

Exactly ! E.g. afaicr Graphviz plugin caches generated drawings due to
the fact that generating them is a time-consuming , resource intensive
task .

... don't know exactly about the json files in this thread though ...

</OT?>

>> The error message written to the log is:
>>
>> Trac[buildrequests] ERROR: Exception occured writing to file: [Errno
>> 2] No such file or directory: 'buildrequests/json/breqs.json'
>> Traceback (most recent call last): File
>> "build\bdist.win32\egg\buildrequests\buildrequests.py", line 109, in
>> process_request file = open(breq_file, 'w') IOError: [Errno 2] No
>> such file or directory: 'buildrequests/json/breqs.json'
>>

... if you ask me ... maybe it's that you are in Windows (<= ouch ! ;)
and you are not using the right path delimiter e.g.
buildrequests/json/breqs.json vs buildrequests\json\breqs.json

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

[paul.wilding]

Thanks for the replies. I put the JSON file in htdocs as once it is written to it will be read from by a JQuery UI library that I am using (it needs to be stored in a file to allow for editing of the data). I see where you're coming from though so where would be best to put the file?

Both of the Itemplates methods are implemented as below:

    def get_templates_dirs(self):

        from pkg_resources import resource_filename

        return [resource_filename(__name__, 'templates')]

    def get_htdocs_dirs(self):

        from pkg_resources import resource_filename

        return [('buildrequests', resource_filename(__name__, 'htdocs'))]

I tried changing the slashes to backward slahes but all that occurred was a different error message:

 

Trac[buildrequests] ERROR: Exception occured writing to file: [Errno 2] No such file or directory: 'buildrequests\\json\x08reqs.json'

Traceback (most recent call last):

  File "build\bdist.win32\egg\buildrequests\buildrequests.py", line 103, in process_request

    file = open(breq_file, 'w')

IOError: [Errno 2] No such file or directory: 'buildrequests\\json\x08reqs.json'

Slight confused as to why it interpreted the name of the file as it did! I use forward slashes to add the scripts and templates to the plugin so I think that's the correct way to go anyway. 

[Cooke, Mark]

> -----Original Message-----
> From: trac-...@googlegroups.com On Behalf Of paul.wilding
> Sent: 18 July 2012 11:21
> To: trac-...@googlegroups.com
> Subject: Re: [Trac] Writing to a JSON file
>
> Thanks for the replies. I put the JSON file in htdocs as once
> it is written to it will be read from by a JQuery UI library
> that I am using (it needs to be stored in a file to allow for
> editing of the data). I see where you're coming from though
> so where would be best to put the file?
>
> Both of the Itemplates methods are implemented as below:
>
> def get_templates_dirs(self):
>
> from pkg_resources import resource_filename
>
> return [resource_filename(__name__,
> 'templates')]
>
>
> def get_htdocs_dirs(self):
>
> from pkg_resources import resource_filename
>
> return [('buildrequests',
> resource_filename(__name__, 'htdocs'))]

I may be barking up the wrong tree but my understanding is that your plugin's `htdocs` is not a real folder, so you cannot write to it (it is a convenience internal redirect into the contents of your egg file)...

I have stored a similar file in the `attachments` folder but that was a data file read only by the plugin. You would probably need to provide your own sub-path of your plugin interface that returns the data file in an appropriate way.

In another case, I used a configurable path in the plugin conf so that the server admin could setup a secure accessible area (may need extra apache conf to allow the download).

Hope that helps,

~ mark c

> --
> You received this message because you are subscribed to the
> Google Groups "Trac Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/trac-users/-/VIZSxikRzZsJ.
> To post to this group, send email to trac-...@googlegroups.com.
> To unsubscribe from this group, send email to
> trac-users+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/trac-users?hl=en.
>
>

[paul.wilding]

Thanks very much - it looks like you were right, using the file on a 'normal' filepath works fine!

> trac-users+unsubscribe@googlegroups.com.

[RjOllos]

On Tuesday, July 17, 2012 10:57:57 PM UTC-4, olemis wrote:

> As a side-note I think it's poor design to write into htdocs, since this
> is meant to be served by web servers as static, cache-able content.
>

<OT?>

Exactly ! E.g. afaicr Graphviz plugin caches generated drawings due to
the fact that generating them is a time-consuming , resource intensive
task .

Right, both GraphvizPlugin and PlantUmlMacro do this, so they should provide good examples. However, they write in $TRACENV/cache, or a similarly named directory.

[RjOllos]

On Wednesday, July 18, 2012 6:21:24 AM UTC-4, paul.wilding wrote:

Thanks for the replies. I put the JSON file in htdocs as once it is written to it will be read from by a JQuery UI library that I am using (it needs to be stored in a file to allow for editing of the data). I see where you're coming from though so where would be best to put the file?

I'm not entirely sure what you are trying to do, but it sounds like it might be similar to the AutcompleteUsersPlugin, which serves data to a jQuery UI plugin through a `/subjects` page. http://trac-hacks.org/wiki/AutocompleteUsersPlugin

It sounds like you have it working though, so I'd be interested if you have found a different or better way. What data are you trying to provide to jQuery UI? Do you plan to publish your code eventually to Trac-hacks?

[paul.wilding]

At my company we have sets of tickets which are grouped as build requests for a particular version of our software. The details of these build requests are stored in a separate table in the trac database which I'm developing this plugin to display and allow for it to be manipulated so I need to get it from the database to the UI.

I got it to work by simply putting the file in my local area for now (clearly this won't work in the long run). However I'm now looking at using the xmlrpc plugin instead as this means I don't have to bother reading/writing to a file and the UI I'm using (kendo UI) can remotely access data in XML or JSON. 

Unfortunately I won't be able to publish the code to trac-hacks in the end as it will be copyrighted to the company and also due to the use of a separate custom table I don't see it being beneficial to many people!

[RjOllos]

You may want to take a look at DuplicateTicketSearchPlugin, which takes the approach of accessing data in JavaScript through the RPC interface. I'm still trying to understand how it works.

http://trac-hacks.org/wiki/DuplicateTicketSearchPlugin

[Benjamin Lau]

Hmmm... that actually sounds pretty interesting. Where I work I
devised a similar system (but using branches in our git repository).
You make sure your branch is named the right way and a build script
will automatically merge all the branches that belong to that version.
Anything that doesn't merge cleanly gets kicked out and it then
generates a notice about what branches merged or failed to merge along
with a combined change log and details about the merge failures.

Ben

> --
> You received this message because you are subscribed to the Google Groups
> "Trac Users" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/trac-users/-/hKMrHVbfp-QJ.

>
> To post to this group, send email to trac-...@googlegroups.com.
> To unsubscribe from this group, send email to

> trac-users+...@googlegroups.com.

[paul.wilding]

Thanks, I've been using this as a loose model and am slowly getting to grips with how it works. I have a further problem now which I've posted here:

http://stackoverflow.com/questions/11651649/trac-json-rpc-issue-when-using-jquery-ajax 

I don't know whether anyone has any ideas for this?

[Jun Omae]

Hi,

On Wed, Jul 25, 2012 at 11:10 PM, paul.wilding
<paul.wi...@googlemail.com> wrote:
> Thanks, I've been using this as a loose model and am slowly getting to grips
> with how it works. I have a further problem now which I've posted here:
> http://stackoverflow.com/questions/11651649/trac-json-rpc-issue-when-using-jquery-ajax
>
> I don't know whether anyone has any ideas for this?

Please try use `dataType: "json"` instead of "jsonp".

--
Jun Omae <jun...@gmail.com> (大前 潤)

[paul.wilding]

I’ve changed that and also was advised on stack overflow to change my data field to look like this:

data: JSON.stringify({method: "breq.getBreqs"})

 

My error message now reads as this:

Trac[json_rpc] ERROR: RPC(json) decode error

Traceback (most recent call last):

  File "build\bdist.win32\egg\tracrpc\json_rpc.py", line 148, in parse_rpc_request
    data = json.load(req, cls=TracRpcJSONDecoder)
  File "C:\Python27\Lib\json\__init__.py", line 278, in load
    **kw)
  File "C:\Python27\Lib\json\__init__.py", line 339, in loads
    return cls(encoding=encoding, **kw).decode(s)
  File "build\bdist.win32\egg\tracrpc\json_rpc.py", line 99, in decode
    obj = json.JSONDecoder.decode(self, obj, *args, **kwargs)
  File "C:\Python27\Lib\json\decoder.py", line 369, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 1 - line 1 column 133 (char 1 - 133)
Trac[web_ui] ERROR: RPC(JSON-RPC) Error

Traceback (most recent call last):

  File "build\bdist.win32\egg\tracrpc\web_ui.py", line 143, in _rpc_process
    rpcreq = req.rpc = protocol.parse_rpc_request(req, content_type)
  File "build\bdist.win32\egg\tracrpc\json_rpc.py", line 162, in parse_rpc_request
    raise JsonProtocolException(e, -32700)
JsonProtocolException: Extra data: line 1 column 1 - line 1 column 133 (char 1 - 133)
Trac[json_rpc] DEBUG: RPC(json) encoded response: {"error": {"message": "JsonProtocolException details : Extra data: line 1 column 1 - line 1 column 133 (char 1 - 133)", "code": -32700, "name": "JSONRPCError"}, "result": null, "id": null}
 
Trac[main] DEBUG: Dispatching <Request "GET '/chrome/common/trac_logo_mini.png'">
Trac[main] DEBUG: Dispatching <Request "POST '/jsonrpc'">
Trac[api] INFO: Synchronized '' repository in 0.00 seconds
Trac[web_ui] DEBUG: RPC incoming request of content type 'application/json' dispatched to <tracrpc.json_rpc.JsonRpcProtocol object at 0x06BD40D0>
Trac[web_ui] DEBUG: RPC(JSON-RPC) call by 'PaulWilding'
Trac[json_rpc] ERROR: RPC(json) decode error

Traceback (most recent call last):

  File "build\bdist.win32\egg\tracrpc\json_rpc.py", line 148, in parse_rpc_request
    data = json.load(req, cls=TracRpcJSONDecoder)
  File "C:\Python27\Lib\json\__init__.py", line 278, in load
    **kw)
  File "C:\Python27\Lib\json\__init__.py", line 339, in loads
    return cls(encoding=encoding, **kw).decode(s)
  File "build\bdist.win32\egg\tracrpc\json_rpc.py", line 99, in decode
    obj = json.JSONDecoder.decode(self, obj, *args, **kwargs)
  File "C:\Python27\Lib\json\decoder.py", line 369, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 1 - line 1 column 133 (char 1 - 133)
Trac[web_ui] ERROR: RPC(JSON-RPC) Error

Traceback (most recent call last):

  File "build\bdist.win32\egg\tracrpc\web_ui.py", line 143, in _rpc_process
    rpcreq = req.rpc = protocol.parse_rpc_request(req, content_type)
  File "build\bdist.win32\egg\tracrpc\json_rpc.py", line 162, in parse_rpc_request
    raise JsonProtocolException(e, -32700)
JsonProtocolException: Extra data: line 1 column 1 - line 1 column 133 (char 1 - 133)
Trac[json_rpc] DEBUG: RPC(json) encoded response: {"error": {"message": "JsonProtocolException details : Extra data: line 1 column 1 - line 1 column 133 (char 1 - 133)", "code": -32700, "name": "JSONRPCError"}, "result": null, "id": null}

 

I’ve tried logging the request in parse_rpc_request in various ways but no output has been produced from this so I haven’t been able to inspect the request. I’ve used self.log.debug(str(req)), self.log.debug(json.load(req)) and self.log.debug(str(json.load(req))) to try to do this. I don’t know whether there’s another way to log this or if you have any idea of what the problem might be now?