twitter extension: Able to insert a return/new line?

[Chris Alemany]

Hi all,

I'm setting up the twitter extension. It works great!

One nice-to-have would be if I could insert a new line into the format.

I've tried \n with no success.  

Suggestions?

Chris

[Chris Alemany]

And any idea how I could add the barometer trend value?

I'm still learning how to find all of the available values in extensions and throughout the system.

Thanks all!

www.alberniweather.ca

@alberniweather

[vince]

On Wednesday, December 27, 2017 at 12:02:24 PM UTC-8, Chris Alemany wrote:

And any idea how I could add the barometer trend value?

I'm still learning how to find all of the available values in extensions and throughout the system.

https://github.com/weewx/weewx/wiki/twitter has the docs

Speculating on your other question, wondering if 'format' would let you add the newline. 

I like your system uptime (what a stable system :-)

Server uptime: 17527 days, 21 hours, 15 minutes

[Chris Alemany]

Haha. I suspect that 'server uptime' variable is optimistic. :)

Maybe this is a more general question than just on this extension.

How do I know what weewx values are available to the extension (or any extension for that matter?).  For example, what is the "rain" variable actually returning in twitter.  It is not the daily rain sum... how can I know what is what and what is available for display?

Cheers,

Chris

[mwall]

On Thursday, December 28, 2017 at 2:19:02 PM UTC-5, Chris Alemany wrote:

Haha. I suspect that 'server uptime' variable is optimistic. :)

Maybe this is a more general question than just on this extension.

How do I know what weewx values are available to the extension (or any extension for that matter?).  For example, what is the "rain" variable actually returning in twitter.  It is not the daily rain sum... how can I know what is what and what is available for display?

the set of observations depends on what hardware is running and what services are running.

start with the hardware docs:

http://weewx.com/docs/hardware.htm

see the 'station data' section for your hardware.

then see the docs for the services you are running.  for example, StdWXCalculate adds a few:

http://weewx.com/docs/usersguide.htm#StdWXCalculate

but the fastest and most accurate way is rather pythonic: just do it!  run weewx directly and see what it spits out in each LOOP and REC dictionary.

m

[Chris Alemany]

On Thursday, December 28, 2017 at 11:58:22 AM UTC-8, mwall wrote:

but the fastest and most accurate way is rather pythonic: just do it!  run weewx directly and see what it spits out in each LOOP and REC dictionary.

m

I sense this is the answer I am looking for (Luke lol) but as to how I might accomplish that... my use of the pythonic-force is weak.

Help me Obi-MWall-Kenobi you're my only hope. 

:) 

[mwall]

On Thursday, December 28, 2017 at 3:24:00 PM UTC-5, Chris Alemany wrote:

On Thursday, December 28, 2017 at 11:58:22 AM UTC-8, mwall wrote:

but the fastest and most accurate way is rather pythonic: just do it!  run weewx directly and see what it spits out in each LOOP and REC dictionary.

I sense this is the answer I am looking for (Luke lol) but as to how I might accomplish that... my use of the pythonic-force is weak.

Help me Obi-MWall-Kenobi you're my only hope. 

here you go:

http://weewx.com/docs/usersguide.htm#running 

m

[Chris Alemany]

Whoa. OK, that was awesome.

However, if I put in 'dayRain' (or dayrain) as it is delivered within the LOOP or REC packets from weewx within my format command for the twitter extension, it just spits out:

 Rain Today: {dayRain:%.1f}mm  

rather than actually parsing the variable.  

However, other variables, like outTemp or heatindex or windchill seem to match up and work just fine.  So I am definitely missing a crucial step here.

Chris

[Chris Alemany]

Also, any ideas on how to pass a new line within the format option so that

Output

on

Twitter

can look

more

like this?

[mwall]

On Thursday, December 28, 2017 at 4:12:38 PM UTC-5, Chris Alemany wrote:

Whoa. OK, that was awesome.

However, if I put in 'dayRain' (or dayrain) as it is delivered within the LOOP or REC packets from weewx within my format command for the twitter extension, it just spits out:

 Rain Today: {dayRain:%.1f}mm  

rather than actually parsing the variable.  

However, other variables, like outTemp or heatindex or windchill seem to match up and work just fine.  So I am definitely missing a crucial step here.

are the observations you want in REC or LOOP?

use the 'binding' option to the weewx-twitter extension to choose which you want to tweet

if that does not get what you want, then you'll have to modify the twitter.py code to make additional database queries (see other restful extensions in restx.py for examples of how to do this)

[mwall]

On Thursday, December 28, 2017 at 4:24:05 PM UTC-5, Chris Alemany wrote:

Also, any ideas on how to pass a new line within the format option so that

Output

on

Twitter

can look

more

like this?

use a newline?  maybe:

format = "first line\\nsecond line\\nthird line"

or if that does not work, try to find the unicode character for newline and stick that into your string

m 

[Chris Alemany]

Hmm.  

On the variabiles:

I am using the default binding (archive).  I don't need the frequency of loop, though, for debugging sake I should mention that when I have tried out binding=loop, weewx fails to start. I have not looked into why that might be as it's not part of my goal.

This is what weewx reports from the Vantage raw stream:

REC:    2017-12-28 12:50:00 PST (1514494200) altimeter: 29.8871528188, appTemp: 32.1224062668, barometer: 29.901, cloudbase: 101.0, consBatteryVoltage: 4.62, dateTime: 1514494200, dayET: 0.0, dayRain: 0.75, dewpoint: 35.2, ET: 0.0, extraAlarm1: 0.0, extraAlarm2: 0.0, extraAlarm3: 0.0, extraAlarm4: 0.0, extraAlarm5: 0.0, extraAlarm6: 0.0, extraAlarm7: 0.0, extraAlarm8: 0.0, forecastIcon: 7.0, forecastRule: 93, heatindex: 35.2, highOutTemp: 35.2, highRadiation: 58.0, highUV: 0.0, humidex: 35.2, inDewpoint: 40.2825325477, inHumidity: 38.0, insideAlarm: 0.0, inTemp: 66.7, interval: 5, leafWet4: 0.0, lowOutTemp: 35.1, maxSolarRad: 200.328662136, monthET: 0.22, monthRain: 3.13, outHumidity: 100.0, outsideAlarm1: 0.0, outsideAlarm2: 0.0, outTemp: 35.2, outTempDay: 35.2, outTempNight: None, pressure: 29.7870205707, radiation: 48.0, rain: 0.0, rainAlarm: 0.0, rainRate: 0.0, rxCheckPercent: 97.375, soilLeafAlarm1: 0.0, soilLeafAlarm2: 0.0, soilLeafAlarm3: 0.0, soilLeafAlarm4: 0.0, stormRain: 0.94, stormStart: 1514188800.0, sunrise: 1514477640.0, sunset: 1514507280.0, trendIcon: -60.0, txBatteryStatus: 0, usUnits: 1, UV: 0.0, windchill: 35.2, windDir: None, windGust: 0.0, windGustDir: None, windrun: 0.000566893424036, windSpeed: 0.0, windSpeed10: 0.0, yearET: 32.26, yearRain: 28.79

LOOP:   2017-12-28 12:50:27 PST (1514494227) altimeter: 29.8871528188, appTemp: 32.1224062668, barometer: 29.901, cloudbase: 101.0, consBatteryVoltage: 4.62, dateTime: 1514494227, dayET: 0.0, dayRain: 0.75, dewpoint: 35.2, extraAlarm1: 0, extraAlarm2: 0, extraAlarm3: 0, extraAlarm4: 0, extraAlarm5: 0, extraAlarm6: 0, extraAlarm7: 0, extraAlarm8: 0, forecastIcon: 7, forecastRule: 93, heatindex: 35.2, humidex: 35.2, inDewpoint: 40.2825325477, inHumidity: 38.0, insideAlarm: 0, inTemp: 66.7, leafWet4: 0.0, maxSolarRad: 200.155823598, monthET: 0.22, monthRain: 3.13, outHumidity: 100.0, outsideAlarm1: 0, outsideAlarm2: 0, outTemp: 35.2, outTempDay: 35.2, outTempNight: None, pressure: 29.7870205707, radiation: 62.0, rain: 0.0, rainAlarm: 0, rainRate: 0.0, soilLeafAlarm1: 0, soilLeafAlarm2: 0, soilLeafAlarm3: 0, soilLeafAlarm4: 0, stormRain: 0.94, stormStart: 1514188800, sunrise: 1514477640, sunset: 1514507280, trendIcon: -60, txBatteryStatus: 0, usUnits: 1, UV: 0.0, windchill: 35.2, windDir: None, windGust: 0.0, windGustDir: None, windSpeed: 0.0, windSpeed10: 0.0, yearET: 32.26, yearRain: 28.79

I used dayrain without success.

On the newline issue.  No luck with either of the escape characters, which I had tried before, or the unicode.

I have also tried a few techniques for getting new lines into code listed here:

https://www.smallsurething.com/multi-line-strings-in-python/

Unfortunately, no success.

Chris

[mwall]

On Thursday, December 28, 2017 at 6:52:36 PM UTC-5, Chris Alemany wrote:

I used dayrain without success.

at line 267 of twitter.py change this:

    def process_record(self, record, dummy_manager):

        if self.unit_system is not None:

            record = weewx.units.to_std_system(record, self.unit_system)

        record['station'] = self.station

to this:

    def process_record(self, record, mgr):

        record = self.get_record(record, mgr)

        if self.unit_system is not None:

            record = weewx.units.to_std_system(record, self.unit_system)

        record['station'] = self.station

then restart weewx.

m 

[Chris Alemany]

no improvement. :(

[Chris Alemany]

This is the format string:

format = "{station} {dateTime:%H:%M %b %d %Y}: T: {outTemp:%.1f}ºC D: {dewpoint:%.1f}ºC W: {windDir:ORD} {windSpeed:%.1f}kph gusting to {windGust:%.1f}kph H: {outHumidity:%.0f}% P: {barometer:%.2f}hPa Rain Rate: {rainRate:%.1f}mm/hr Chill:{windchill:%.1f}ºC HIndex:{heatindex:%.1f}ºC Solar: {radiation:%.0f}W/m2 UV: {UV:%.0f} #PortAlberni https://www.alberniweather.ca https://www.youtube.com/alberniweather"

This is what twitter output looks like (same for dayRain or dayrain as variable):

Hourly Auto-Tweet 16:45 Dec 28 2017: T: 2.4ºC D: 2.4ºC W: * 0.0kph gusting to 0.0kph H: 100% P: 1013.07hPa Rain: {dayrain:%.1f}mm Rain Rate: 0.0mm/hr Chill:2.4ºC HIndex:2.4ºC Solar: 0W/m2 UV: 0 #PortAlberni https://www.alberniweather.ca 

and

Hourly Auto-Tweet 16:40 Dec 28 2017: T: 2.5ºC D: 2.5ºC W: * 0.0kph gusting to 0.0kph H: 100% P: 1013.04hPa Rain: {dayRain:%.1f}mm Rain Rate: 0.0mm/hr Chill:2.5ºC HIndex:2.5ºC Solar: 0W/m2 UV: 0 #PortAlberni https://www.alberniweather.ca 

[Thomas Keffer]

format = "{station} {dateTime:%H:%M %b %d %Y}: T: {outTemp:%.1f}ºC D: {dewpoint:%.1f}ºC W: {windDir:ORD} {windSpeed:%.1f}kph gusting to {windGust:%.1f}kph H: {outHumidity:%.0f}% P: {barometer:%.2f}hPa Rain Rate: {rainRate:%.1f}mm/hr Chill:{windchill:%.1f}ºC HIndex:{heatindex:%.1f}ºC Solar: {radiation:%.0f}W/m2 UV: {UV:%.0f} #PortAlberni https://www.alberniweather.cahttps://www.youtube.com/alberniweather"

I'm not seeing dayrain (or dayRain) in your format at all. Where is it coming from?

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Alemany]

Apologies I had removed it temporarily.... it is supposed to be before Rain Rate.

format = "{station} {dateTime:%H:%M %b %d %Y}: T: {outTemp:%.1f}ºC D: {dewpoint:%.1f}ºC W: {windDir:ORD} {windSpeed:%.1f}kph gusting to {windGust:%.1f}kph H: {outHumidity:%.0f}% P: {barometer:%.2f}hPa Rain {dayrain:%.1f}mm Rain Rate: {rainRate:%.1f}mm/hr Chill:{windchill:%.1f}ºC HIndex:{heatindex:%.1f}ºC Solar: {radiation:%.0f}W/m2 UV: {UV:%.0f} #PortAlberni https://www.alberniweather.ca https://www.youtube.com/alberniweather"

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

[Thomas Keffer]

You're not making it easy!

Make sure you're showing us the correct format and the exact output that goes along with it. Better yet, cut and paste the whole twitter section of your weewx.conf.

Python is case-sensitive, so you definitely want dayRain.

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.

[Chris Alemany]

Thanks for the mental push on ‘the whole twitter section”. I’ve figured out the day rain issue. 

I was using METRIC rather than METRICWX so dayRain was reporting in cm rather than mm.  Since it was a small amount of rain… it was reporting 0.0cm.  We’ve subsequently had almost 20mm of rain today, which means there is enough to notice the units mixup.

Here is the successful wview.conf including the formatting used for the tweet below:

station = Hourly Auto-Tweet

unit_system = METRICWX

post_interval = 3600

format_None = *

format = "{station} {dateTime:%H:%M %b %d %Y}: T: {outTemp:%.1f}ºC D: {dewpoint:%.1f}ºC W: {windDir:ORD} {windSpeed:%.1f}kph gusting to {windGust:%.1f}kph H: {outHumidity:%.0f}% P: {barometer:%.2f}hPa Rain {dayRain:%.2f}mm Rain Rate: {rainRate:%.1f}mm/hr Chill:{windchill:%.1f}ºC HIndex:{heatindex:%.1f}ºC Solar: {radiation:%.0f}W/m2 UV: {UV:%.0f} #PortAlberni https://www.alberniweather.ca https://www.youtube.com/alberniweather"

https://twitter.com/Auto_AlberniWx/status/946581641299771392

So that is good.

Unfortunately, I have not found a solution to the original request of inserting a new line. But I am willing to let it go and chalk it up to a limitation of twython. :)

Thank you all for your help and suggestions.

Chris

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/YXlx_CAWO5Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

[Andrew Milner]

Try and insert 
 for the newline

Chris

[Chris Alemany]

No luck. Not parsed. :(

Hourly Auto-Tweet 09:35 Dec 29 2017: 
 T: 2.2ºC D: 2.2ºC W: * 0.0kph gusting to 0.0kph H: 100% P: 1013.92hPa Rain 0.25mm Rain Rate: 0.0mm/hr Chill:2.2ºC HIndex:2.2ºC Solar: 10W/m2 UV: 0

[Andrew Milner]

I think I got it wrong and it should be &#16 rather than &#010 and also I would have thought more like

format = "{station} {dateTime:%H:%M %b %d %Y&#010}: T: {outTe
or

format = "{station} {dateTime:%H:%M %b %d %Y}: T: {outTe

really clutching at straws though!!

good luck

[gjr80]

Just to humour me could we try a URL encoded line feed:

%0a

Gary

[Chris Alemany]

That's a negative on both  and %0a

It just doesn't seem to want to parse any of the characters in the strings. I did some searching around and it certainly looks like, from examples like this:

https://gist.github.com/dangayle/c211c1af232974f00564

def tweet(sentence):

    """Tweet sentence to Twitter."""

    try:

        sys.stdout.write("{} {}\n".format(len(sentence), sentence))

        twitter.update_status(status=sentence)

    except:

        pass

That twython is capable of sending "new lines" to twitter. And of course python is able to have new lines within strings.  Is it perhaps some way in how the .conf file is being parsed that does not allow these characters through properly?

Just throwing that out there as an idea, no idea whatsoever if it has any merit. :)

Thanks for all the suggestions.

[vince]

On Saturday, December 30, 2017 at 11:58:47 AM UTC-8, Chris Alemany wrote:

That's a negative on both  and %0a

That twython is capable of sending "new lines" to twitter. And of course python is able to have new lines within strings.  Is it perhaps some way in how the .conf file is being parsed that does not allow these characters through properly?

This is ugly but seems to work.....

>>> mytweet=("this is line1\n" "this is line2\n" "this is line3\n")

>>> print mytweet

this is line1

this is line2

this is line3

[Chris Alemany]

Doesn't seem to parse when used in the format option in weewx.conf just spits out the characters it is given...  I'll look into mangling the twitter.py file perhaps.

@Auto_AlberniWx

[Chris Alemany]

Success!

I shifted focus to the _DEFAULT_FORMAT value in the twitter.py file directly rather than the weewx.conf file and the first and simplest option works!

The \n escape sequence works to create a new line.

This:

 _DEFAULT_FORMAT = '{station} {dateTime:%H:%M %b %d %Y}:\nT: {outTemp:%.1f}C\nD: {dewpoint:%.1f}C\nW: {windDir:ORD} {windSpeed:%.1f}kph gusting to {windGust:%.1f}kph\nH: {outHumidity:%.0f}%\nP: {barometer:%.2f}hPa\nDay Rain {dayRain:%.2f}mm\nRain Rate: {rainRate:%.1f}mm/hr\nChill:{windchill:%.1f}C\nHIndex:{heatindex:%.1f}C\nSolar: {radiation:%.0f}W/m2\nUV: {UV:%.0f}\n#PortAlberni\nhttps://www.alberniweather.ca\nhttps://www.youtube.com/alberniweather'

Produced this:

Hourly Auto-Tweet 21:25 Dec 30 2017:

T: 2.4C

D: 0.7C

W: NNE 1.5kph gusting to 3.1kph

H: 88%

P: 1028.18hPa

Day Rain 3.30mm

Rain Rate: 0.0mm/hr

Chill:2.4C

HIndex:2.4C

Solar: 0W/m2

UV: 0

#PortAlberni

https://www.alberniweather.ca

I had degree symbols in the string originally, which killed weewx when I tried to use them in the DEFAULT value. So I took those out and it works fine now. I'll see if I can use a character encoding to get the degree symbol, but it's not a deal breaker.  The main goal has been fulfilled, even if it is a little hack-ish :)

BTW: I also thought it might have been a double quote vs. single quote problem in the weewx.conf but still no love when I tried that, so I will settle on this solution.

Thanks again for everyones help.  I will post again if I get degree symbols in there. :)

Cheers,

Chris

[Andrew Milner]

Just a thought - is it possible that in .conf there should not be any quotes around the format string?  Other values appear to have no quotes around them.

[Andrew Milner]

A further thing - significance unknown - looking at the wiki the examples in there seem to include semicolons between items, and I cannot see the significance of format_None = * mentioned either.

Perhaps something more like:

station = Hourly Auto-Tweet

unit_system = METRICWX

post_interval = 3600

format = "{station}; \n{dateTime:%H:%M %b %d %Y}; \nT: {outTemp:%.1f}ºC;  \

\nD: {dewpoint:%.1f}ºC; \nW: {windDir:ORD} {windSpeed:%.1f}kph gusting to {windGust:%.1f}kph;  \

\nH: {outHumidity:%.0f}%; \nP: {barometer:%.2f}hPa; \nRain {dayRain:%.2f}mm; Rain Rate: {rainRate:%.1f}mm/hr;  \

\nChill:{windchill:%.1f}ºC; \nHIndex:{heatindex:%.1f}ºC; \nSolar: {radiation:%.0f}W/m2; \nUV: {UV:%.0f};  \

\n#PortAlberni https://www.alberniweather.ca https://www.youtube.com/alberniweather"

would also work??.....

[Chris Alemany]

Thanks for this.

No luck. It appears the direct editing of the default format variable in twitter.py is best.

[gjr80]

On Tuesday, 2 January 2018 06:58:07 UTC+10, Chris Alemany wrote:

It appears the direct editing of the default format variable in twitter.py is best.

Perhaps not, if only we had turned on debug and looked at the log...

using the format config option in weewx.conf (excuse the station name):

Jan  2 13:05:16 stretch22 weewx[2871]: restx: Twitter: msg: Narangba Test - weeWX 3.8.0a test2 13:05 Jan 02 2018:\nT: 32.6C\nD: 27.2C\nW: N 0.0kph gusting to 0.0kph\nH: 80%\nP: 31.10hPa\nDay Rain {dayRain:%.2f}mm\nRain Rate: 0.0mm/hr\nChill:32.6C\nHIndex:32.6C\nSolar: 924W/m2\nUV: 13\n#PortAlberni\nhttps://www.alberniweather.ca\nhttps://www.youtube.com/alberniweather

omitting the format config option and overriding _DEFAULT_FORMAT with the exact same format string (again excuse the station name):

Jan  2 12:55:16 stretch22 weewx[2744]: restx: Twitter: msg: Narangba Test 12:55 Jan 02 2018:#012T: 32.6C#012D: 27.1C#012W: N 0.0kph gusting to 0.0kph#012H: 80%#012P: 31.10hPa#012Day Rain {dayRain:%.2f}mm#012Rain Rate: 0.0mm/hr#012Chill:32.6C#012HIndex:32.6C#012Solar: 948W/m2#012UV: 13#012#PortAlberni#012https://www.alberniweather.ca#012https://www.youtube.com/alberniweather

The logged line is the string that is passed to Twython to post as a tweet.

The issue is that when parsing a config file configobj does not convert \n to a linefeed escape sequence but rather it escapes the \n (ie \\n) so that we end up with a literal \n sequence in the returned string (more generally configobj converts any escape sequence to literal characters - which is why using #0a in the format config option did not work either) . Twython, twitter or whatever ultimately URL encodes this and we end up with a plain \ followed by a n, ie no linefeed. On the other hand, _DEFAULT_FORMAT is a python string with embedded \n escape sequence and the escape sequence flows through twitter.py and Twython or twitter URL encodes this escape sequence as a linefeed and consequently you get the formatted tweet you seek. A solution that allows the use of a user defined format config option that supports linefeeds is to intercept the format in the config dict returned by  configobj and replace the literal \n sequence with \n escape sequence back in. It feels a little hackish to me as it is \n specific but the adding the highlighted lines to twitter.py (circa line 180) should do the trick:

        if usn is not None:
            site_dict['unit_system'] = weewx.units.unit_constants[usn]
            loginf('units will be converted to %s' % usn)

        try:
            site_dict['format'] = site_dict['format'].replace('\\n','\n')
        except:
            pass
        site_dict.setdefault('format', self._DEFAULT_FORMAT)
        site_dict.setdefault('format_None', self._DEFAULT_FORMAT_NONE)
        site_dict.setdefault('format_utc', False)

Of course _DEFAULT_FORMAT should be set back to its original setting and the format config option under [[Twitter]] now supports the use of \n to insert a new line in the tweet. Note there is no need to enclose the format config option in any quotes and of course weeWX will need to be restarted for the changes to take effect.

I will caveat this with I do not use Twitter nor have I tested this on Twitter.

Gary

[Thomas Keffer]

If the problem is configobj, then use triple quotes, without the \n:

format = """{station} {dateTime:%H:%M %b %d %Y}: 

T: {outTemp:%.1f}ºC 

D: {dewpoint:%.1f}ºC"""

-tk

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.

[gjr80]

Ah, I was tried triple quotes but with /n. But yes, configobj is the issue.

Gary