Scheduled and Unexpected Server Down Time Annoucements

[mytaglist.com support]

In this thread we will announce scheduled and unexpected server down time.

• Wireless Sensor Tag Web Service automatically restarts everyday at 7AM UTC (midnight at Pacific Time) and HTTP requests during restart is queued until restart finishes (in 30~60 seconds); This means you may receive temporary "Timeout" error between 7:00~7:02AM everyday when using the app or Web interface. If you try again in a few minutes everything will be back to normal.
  
  
• About once every 3 months, the Web server may have to reboot due to "Windows Update" that installs important security updates. The reboot takes anywhere from 30 seconds to 3 minutes. During that time you will not be able to use the app or Web interface; however because Tag Managers will queue locally up to ~40 transmissions from tags when server cannot be reached, you will not lose any temperature log data or miss any motion/temperature events.
  

[mytaglist.com support]

We expect that the Wireless Tag Cloud Service be temporarily unavailable for up to 5 minutes any time during 

6:00am CDT on Sunday, September 28, 2014 to 6:00am CDT on Monday, September 29, 2014

because of server reboot enforced by Rackspace. Please check 

https://status.rackspace.com/

for detailed status and schedule of the service availability. Our service is located in Chicago ("ORD"), and relies on "Cloud Servers (Next Generation)" and "Cloud Block Storage" to function.

[mytaglist.com support]

A 3rd case when the Web Service becomes temporarily unavailable is when some user are actively using the app at the instant we updates/patches the server software. 

When this happens, the server has to be restarted. This takes about 30 seconds ~ 60 seconds to complete, similar to our daily restarts. 

If no user is actively using the app at the instant we update the server software, no service interruption will occur. 

Normally server software update happens less than once per week. But during/after new feature releases, it is more frequent, 2~3 times a day.

On Tuesday, August 19, 2014 10:23:40 AM UTC-7, mytaglist.com support wrote:

[mytaglist.com support]

KumoApp Engine is currently under maintenance. It will not affect the operation of other part of the system. However, server restart may be needed a couple of times (which can create 2~3 minute service outage). 

[mytaglist.com support]

KumoApp Engine has been restored.
Since back-in-range notification is also hosted inside KumoApp Engine, the back-in-range notification was not working while KumoApp Engine was taken offline, but is now back.

[mytaglist.com support]

KumoApp Engine is down for repair again. We are working as quickly as possible to get this back running again. 

[mytaglist.com support]

KumoApp engine is now running smoothly. 

It now has more efficient core to use less memory and runs with less latency than before.

[mytaglist.com support]

We are restarting the service now in order to install new component to support the change on the Apple's push notification server, in which they are now dropping support for SSL and requiring TLS. This has caused iOS push notification to not function since early this morning. Wireless tag service will be back in a few minutes.  

[mytaglist.com support]

From about 3PM PST to 10PM PST today (Jan 31)  KumoApp engine was not always running because of an issue with Dropcam feature of the KumoApp. 

This issue was caused by file access permission configuration error after we moved KumoApp engine to a different server recently. 

This issue is now fixed and KumoApp engine is up and running again.

[vbo...@gmail.com]

Just a few thoughts, 1) Linux anyone? 2) Clustered servers so you have no blackouts, and 3) Open source (DD-WRTable) Tag Manager.

[mytaglist.com support]

Today we are performing maintenance on the temperature log database table. We will be

- Adding more disks

- De-frag/rebuild indexes to allow faster plotting of data

- Add index on the date column to allow:

  - moving data older than 1 year to newly added disks to reduce the size of live temperature data table to allow faster plotting and faster tag manager upload. 

You will still be able to plot data older than 1 year, however plotting performance may be slow until these operations are complete. 

Because of the sheer size of temperature log, some of these operations can take up to 10 minutes. We try to split each operation into small pieces and run them little by little over the next 24 hours. 

You may experience timeout error or slow server response today, but these should be temporary and last no longer than 10 minutes. 

 

[Cao Gadgets]

As of 10PM April 13 this maintenance is completely finished and all service health monitoring shows good results:

- Tag manager uploads/ping finishes within 0.2~0.3 seconds on average (the time orange arrow is lit, when tag manager cannot process user command), vs. 0.4~0.5 seconds before maintenance.

- IFTTT channel trigger check timeout / action error rate within 1% according to IFTTT

- KumoApp engine script starts/stops with normal delay. We fixed an error which caused one KumoApp not to start when Run button is clicked while another KumoApp is starting.

- Back in range check for all out-of-range tags finishes on time with normal level of concurrency.

- Disk load is spread evenly across 5 server SSD disks (3 disks w/ one handling majority of load before maintenance). 

- Server CPU usage <20% vs. 40~50% before maintenance.

[jbtibor]

https://www.mytaglist.com/eth/index.html reports Service unavailable and Kumo Apps not running either.

[Zhiheng Cao]

Server is very slow or down unexpectedly since this morning. We are working on it. 

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-...@googlegroups.com.
To post to this group, send email to wireless-s...@googlegroups.com.
Visit this group at http://groups.google.com/group/wireless-sensor-tags.
To view this discussion on the web visit https://groups.google.com/d/msgid/wireless-sensor-tags/7015ef52-2bfe-471b-b0b2-c9051de0b194%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[mytaglist.com support]

We have found the cause to this problem and fixed it. There was nothing wrong with our code, we just had to enable IPv6 connector in the SQL server configuration manager. Apparently, as connection to the database machine increases above certain number some connections are made to the IPv6 address. 

We apologize for this service interruption. 

On Friday, July 31, 2015 at 2:35:36 PM UTC-7, mytaglist.com support wrote:

Server is very slow or down unexpectedly since this morning. We are working on it. 

On Fri, Jul 31, 2015 at 2:28 PM, jbtibor <jbt...@gmail.com> wrote:

https://www.mytaglist.com/eth/index.html reports Service unavailable and Kumo Apps not running either.

On Tuesday, 14 April 2015 18:31:44 UTC+1, Cao Gadgets wrote:

As of 10PM April 13 this maintenance is completely finished and all service health monitoring shows good results:

- Tag manager uploads/ping finishes within 0.2~0.3 seconds on average (the time orange arrow is lit, when tag manager cannot process user command), vs. 0.4~0.5 seconds before maintenance.

- IFTTT channel trigger check timeout / action error rate within 1% according to IFTTT

- KumoApp engine script starts/stops with normal delay. We fixed an error which caused one KumoApp not to start when Run button is clicked while another KumoApp is starting.

- Back in range check for all out-of-range tags finishes on time with normal level of concurrency.

- Disk load is spread evenly across 5 server SSD disks (3 disks w/ one handling majority of load before maintenance). 

- Server CPU usage <20% vs. 40~50% before maintenance.

On Sunday, April 12, 2015 at 9:58:30 AM UTC-7, mytaglist.com support wrote:

Today we are performing maintenance on the temperature log database table. We will be

- Adding more disks

- De-frag/rebuild indexes to allow faster plotting of data

- Add index on the date column to allow:

  - moving data older than 1 year to newly added disks to reduce the size of live temperature data table to allow faster plotting and faster tag manager upload. 

You will still be able to plot data older than 1 year, however plotting performance may be slow until these operations are complete. 

Because of the sheer size of temperature log, some of these operations can take up to 10 minutes. We try to split each operation into small pieces and run them little by little over the next 24 hours. 

You may experience timeout error or slow server response today, but these should be temporary and last no longer than 10 minutes. 

 

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-tags+unsub...@googlegroups.com.
To post to this group, send email to wireless-sensor-tags@googlegroups.com.

Visit this group at http://groups.google.com/group/wireless-sensor-tags.

[jbtibor]

Still get error on website:

HTTP Error 503. The service is unavailable.

To post to this group, send email to wireless-s...@googlegroups.com.

Visit this group at http://groups.google.com/group/wireless-sensor-tags.

[Matthew M]

Still down for me in AZ.  Tag manager will not connect - the mobile app gives a generic network failure error, and the desktop app does not completely load (see attached file)

. 

[Eric Lee]

Same here in Illinois, tag manager says no Internet. Tried to unplug and replug, same thing.

[Yves Racine]

No service here in Montreal (Canada).... Could you update us when approximately the service will be back (ETA)?

[Zhiheng Cao]

We are still working on it. The problem is that ASP.NET + MSSQL all by itself is suddenly taking much longer to process tag manager data uploads (HTTP POST) since 11AM this morning. We did not make any change to the server. 

We redirected most tag manager uploads to a new web server, so that mytaglist.com (mobile app interface) is back. However tag events still takes much longer (>1 minute each) to upload on the new server. We have tried restarting server, stopping auxiliary services like KumoApp and out of range notification, applying latest patches from Microsoft on the SQL server, etc but they are not helping. We are now adding trace point in the code to narrow down which part of the code is taking longer. 

On Fri, Jul 31, 2015 at 6:12 PM, Yves Racine <yrac...@gmail.com> wrote:

No service here in Montreal (Canada).... Could you update us when approximately the service will be back (ETA)?

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-...@googlegroups.com.

To post to this group, send email to wireless-s...@googlegroups.com.
Visit this group at http://groups.google.com/group/wireless-sensor-tags.

To view this discussion on the web visit https://groups.google.com/d/msgid/wireless-sensor-tags/01541bff-23de-48d1-ad1c-fa97964a6e9f%40googlegroups.com.

[mytaglist.com support]

The service has been restored around 7PM today; We then restarted the KumoApp Engine and after an hour the data upload got very slow again. 

As soon as we stopped the KumoApp Engine it was fast again. 

The KumoApp Engine currently gets triggered by real time tag events using an MSSQL server feature called "Query Notification" which in turn relies on other MSSQL server features such as "Service Broker".  One way to explain what happened today around 11AM that brought down the entire web service could be that a customer started some KumoApps that were utilizing such features in an "unsustainable" way.  Each KumoApp seemed to create its own SQL server connection and the way the number of connection grew was unsustainable.

It is a time for a review of the scalability of the way KumoApp Engine gets real time tag events. It is time to write our own code instead of relying on MSSQL server feature that is overly complicated to know what's wrong when something's wrong. 

Until then the KumoApp feature will be suspended. We will be working as quickly as we can to come up with a more scalable triggering scheme for the KumoApp. 

If you are relying on KumoApp for something, please try to implement it using IFTTT for now. 

- show quoted text -

On Friday, July 31, 2015 at 6:30:56 PM UTC-7, mytaglist.com support wrote:

We are still working on it. The problem is that ASP.NET + MSSQL all by itself is suddenly taking much longer to process tag manager data uploads (HTTP POST) since 11AM this morning. We did not make any change to the server. 

We redirected most tag manager uploads to a new web server, so that mytaglist.com (mobile app interface) is back. However tag events still takes much longer (>1 minute each) to upload on the new server. We have tried restarting server, stopping auxiliary services like KumoApp and out of range notification, applying latest patches from Microsoft on the SQL server, etc but they are not helping. We are now adding trace point in the code to narrow down which part of the code is taking longer. 

On Fri, Jul 31, 2015 at 6:12 PM, Yves Racine <yrac...@gmail.com> wrote:

No service here in Montreal (Canada).... Could you update us when approximately the service will be back (ETA)?

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-tags+unsub...@googlegroups.com.
To post to this group, send email to wireless-sensor-tags@googlegroups.com.

Visit this group at http://groups.google.com/group/wireless-sensor-tags.

[Bradley Britton]

Tag Service is still down

[Jake Rieskamp]

Orange and red light.

[mytaglist.com support]

I found the problem now it is fixed. 

This post describes exactly what was happening and what I did to fix it.

http://stackoverflow.com/questions/10455455/server-2008-rc2-iis-7-5-asp-net-and-requests-queued-poor-performance

The CPU was like 10% but there was 1000+ requests queued. Each request took only 15ms to execute, but including the queued time it took over 1 minute and the tag managers timed out waiting for response. 

Great job Microsoft with the "autoConfig"  of ASP.NET "processModel". 

On Saturday, August 1, 2015 at 7:57:40 PM UTC-7, Jake Rieskamp wrote:

Orange and red light.

[Bradley Britton]

System is down again, all tags lost connection

[Zhiheng Cao]

No it is not down

On Mon, Aug 3, 2015 at 1:20 PM, Bradley Britton <bradbr...@gmail.com> wrote:

System is down again, all tags lost connection

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-...@googlegroups.com.
To post to this group, send email to wireless-s...@googlegroups.com.

Visit this group at http://groups.google.com/group/wireless-sensor-tags.

To view this discussion on the web visit https://groups.google.com/d/msgid/wireless-sensor-tags/17a65db4-985d-42a3-92d0-6c046565332d%40googlegroups.com.

[Bradley Britton]

I have unplugged the tag manager and plugged it back in and it is connected and working. Would there be a reason why my tag manager was red this morning and all my tags were showing no signal. Nothing else I had connected to the internet had lost connection.

[mytaglist.com support]

We are ahead of our schedule. The KumoApp service with better scalability is now operational and in beta state. 

You should see your KumoApps run with faster response/lower delay and higher reliability.

[mytaglist.com support]

We monitor the server hourly and it is also monitored by an automatic system at IFTTT. We get text message when server is having problems and our staff works on it immediately. 

Summary of July 31-Aug 1 server problem:

On last Friday (July 31 from 11AM-7PM) and Saturday some hours the server was essentially down for a combination of a few causes:

- One of the database disk was running low

- More KumoApps were being run creating too many connections to database

- The above caused long database response time for some period, which in turn confused the automatic configuration algorithms on the Web server side to restrict concurrency (thread pool) too much -> long delay even when database delay is back to normal.

By now:

- We changed web server to manual configuration that does not restrict the level of concurrency,

- We completely re-designed how Kumoapp engine gets real-time events so it does not create many database connections even when many users are using it, and

- We now know that disk space even when it is not full, can cause this problem so we will keep monitoring it to ensure always at least there are double amount of empty space as when this happened

Historically, the last time our server was down for this long was Dec 22-23 in year 2013. 

[Simon Ward]

Thanks for working so quickly on this.

[Simon Ward]

[steve]

thanks for all the great work 

[mytaglist.com support]

Below is a notice from Rackspace (hardware and networking provider for wireless tag cloud service) The service outage is only 15-45 seconds but since all internal networking (connection to database) will be down during that time, it is expected that after service is resumed, our server must process a rather large back log and that may take a few minutes to finish. 

On August 24, 2015 from 23:00 CDT to August 25, 2015 at 05:00 CDT, we will perform a scheduled maintenance on the networking infrastructure of our Performance Cloud Servers in the Rackspace ORD Region. This maintenance event will not require any action from you.  Please note the following devices that are associated with this maintenance:

104.130.203.171 / kumoapp
23.253.41.230 / mytaglist.com
 
During this maintenance, network connectivity will be interrupted one time for a period lasting between 15-45 seconds on the Cloud Servers listed above.  This networking interruption will affect all Cloud Servers in this ticket within this region. This includes connectivity from Cloud Servers to Cloud Files, Cloud Load Balancers, Cloud Networking, Cloud Block Storage and RackConnect services, as well as private server-to-server connectivity.
 
Please accept our apologies for the inconvenience.  We always strive to have minimal downtime associated with deploys of upgrades, however, some maintenances require small delays in network traffic while the process completes.  We only perform these types of maintenances after extensive testing and planning to ensure minimal impact.
 
Additional information will be made available via our status page during and after the maintenance at: http://status.rackspace.com.  If you have any questions, please do not hesitate to contact a member of our Fanatical Support Team via live-chat or by phone.
 

Thank You,
 
 
 
Rackspace
Phone Support (US): 1.877.934.0407
Phone Support (INTL): 1.210.581.0407
For US Cloud Customers:
Phone Support (toll free): 1.877.934.0407
Phone Support (toll): 1.210.581.0407
Live Chat: https://mycloud.rackspace.com (select Live Chat)
 
For UK/EMEA Cloud Customers:
Core Cloud: Freephone: 08000 546 345 / Tel: +44 20 8734 4345
Managed Cloud : Freephone: 08000 546 645 / Tel: +44 20 8734 4445

Live Chat: https://mycloud.rackspace.co.uk (select Live Chat)

[mytaglist.com support]

According to Rackspace:

Critical Maintenance: mytaglist.com/wirelesstag.net will be power cycled between10/28/2015, 8:00:00 PM and 10/28/2015, 10:00:00 PM (PDT) in an urgent maintenance.

During the power cycle, the app and web interface will be unavailable for 3 to 5 minutes. 

[mytaglist.com support]

The "comet" feature, i.e. screen auto-update in Web browsers is currently experiencing problems.  This means the status of each tag or KumoApp will not automatically update when there is a change, and you will have to press F5 key or click refresh button.  We are working on solution, ETA 48 hours.  (Note: our iOS app relies on push notification to automatically get the update and will not be affected by this). 

[mytaglist.com support]

The solution has been implemented and the comet feature is working again. 

[mytaglist.com support]

We plan to perform upgrade of backend database at 11PM PST Saturday Feb 13.
During this upgrade, the cloud service will be temporarily unavailable, but we expect this will last for less than 10 minutes. After the upgrade, reliability and response time of the cloud service will improve.

[mytaglist.com support]

We will perform scheduled maintenance on www.mytaglist.com (not my.wirelesstag.net which will be mostly unaffected) on Sunday 9/18 from 11:30PM PDT (6:30AM UTC). 

This should only cause less than 1 minute of down-time. 

We will be moving old data to slower disks to make room for new data. During the file copy the web server and database at mytaglist.com must be temporarily stopped. 

[mytaglist.com support]

We will perform scheduled database disk maintenance on www.mytaglist.com on Friday 11/11 between 10:30PM-11:00PM PST (6:30-7:00AM UTC). 

This may cause up to 20 minutes of down time. During this time, your tag manager may display solid orange (or red) arrow. It will start to locally buffer received sensor data / events. 

After the maintenance is over, the sensor data/ events received during the time should be quickly uploaded to server and you should not see any gap in temperature graphs.

If your tag manager is hosted by my.wirelesstag.net it will not be affected by this maintenance. 

[mytaglist.com support]

We are performing database maintenance currently that affects only plotting graphs/downloading temperature log data (CSV) more than 6 month ago.  This is expected to finish in 30 min to 1 hour. 

Other system functionalities are not affected. 

[mytaglist.com support]

From November 13 8PM PST (November 14  4AM GMT) for about 30 minutes, we will be performing the same kind of database maintenance again which will affect plotting temperature/humidity graphs and CSV download. Other functionality like logging temperature, temperature,humidity,motion notifications, IFTTT recipies etc will not be affected. 

[mytaglist.com support]

We are re-creating one index on mytaglist.com temperature log data currently. Graph viewing/download (except recent 4 month) will be slow until this finishes (in about 30 minutes). 

[mytaglist.com support]

This indexing operation is taking longer than we expected. It may take additional 1 hour to finish. This will affect most attempts to download temperature data from mytaglist.com until it is finished. We apologize for the inconvenience. 

[mytaglist.com support]

Over the last few days, if you have a tag manager hosted by mytaglist.com, you may have experienced long delay (even complete Timed Out error) in user command such as "update now" "beep" or "arm". 

This problem is not due to your tag manager hardware. It is due to a bug in server code that did not handle delay caused when certain valid KumoApp was running. 

This has just been fixed. You should see everything is back to normal with 2-3 second max response time to user commands. 

[mytaglist.com support]

On my.wirelesstag.net, sometimes you may see an error "An established connection was aborted by the software in your host machine" immediately after trying to send a user command.  This appears to be a bug on Google Compute Engine side that SOMETIMES prevents web server on the local machine to connect IRC server on the same machine using its public IP address. The same code running on Rackspace (mytaglist.com) never experiences this problem. 

This has just been fixed by switching to use private IP address when the web server connect to locally running IRC server. 

[Raymond Linschoten]

I'am experiencing the same problem. My KumoApps are not reacting to start or stop anymore.

Op maandag 5 december 2016 18:34:26 UTC+1 schreef Lowell Dykstra:

The KumoApp Server appears to have a problem and is not responding as it should (12-05-16).

Can you look into this please.

[Zhiheng Cao]

Are you using mytaglist.com or my.wirelesstag.net?

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-...@googlegroups.com.
To post to this group, send email to wireless-s...@googlegroups.com.
Visit this group at https://groups.google.com/group/wireless-sensor-tags.

For more options, visit https://groups.google.com/d/optout.

--

Sent from MetroMail

[Raymond Linschoten]

I'am using mytaglist.com but it seems to working again. My KumoApps responding to start and stop again.

Thanks for the fast response!

Op maandag 5 december 2016 21:04:43 UTC+1 schreef mytaglist.com support:

Are you using mytaglist.com or my.wirelesstag.net?

On Mon, Dec 5, 2016 at 12:00 PM Raymond Linschoten <rlins...@gmail.com> wrote:

I'am experiencing the same problem. My KumoApps are not reacting to start or stop anymore.

Op maandag 5 december 2016 18:34:26 UTC+1 schreef Lowell Dykstra:

The KumoApp Server appears to have a problem and is not responding as it should (12-05-16).

Can you look into this please.

On Tuesday, August 19, 2014 at 12:23:40 PM UTC-5, mytaglist.com support wrote:

In this thread we will announce scheduled and unexpected server down time.

• Wireless Sensor Tag Web Service automatically restarts everyday at 7AM UTC (midnight at Pacific Time) and HTTP requests during restart is queued until restart finishes (in 30~60 seconds); This means you may receive temporary "Timeout" error between 7:00~7:02AM everyday when using the app or Web interface. If you try again in a few minutes everything will be back to normal.
  
  
• About once every 3 months, the Web server may have to reboot due to "Windows Update" that installs important security updates. The reboot takes anywhere from 30 seconds to 3 minutes. During that time you will not be able to use the app or Web interface; however because Tag Managers will queue locally up to ~40 transmissions from tags when server cannot be reached, you will not lose any temperature log data or miss any motion/temperature events.
  

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-tags+unsub...@googlegroups.com.

To post to this group, send email to wireless-s...@googlegroups.com.
Visit this group at https://groups.google.com/group/wireless-sensor-tags.
For more options, visit https://groups.google.com/d/optout.

--

Sent from MetroMail

[Raymond Linschoten]

My KumpApps are not responding anymore since this morning, can't start or stop them. There is a notification visible that says "KumoAppEngine had been restarted by administrator for mantenance. Your App should restart automatically when the..." for a few hours now...

Op dinsdag 19 augustus 2014 19:23:40 UTC+2 schreef mytaglist.com support:

[mytaglist.com support]

Between 10-11PM PST, February 11, we will perform server hardware upgrade on my.wirelesstag.net.  

 

First step involves moving all data to a persistent disk drive. Your tag manager will show red up&down arrow and apps and Web Interface will not work for a period of 2~3 minutes during this.

 

Next step, we will shut-down and reboot the server my.wirelesstag.net in order to scale-up the hardware. How long this takes will depend on Google Compute Engine side, but should take no more than 5-10 minutes. Your tag manager will show red up&down arrow and apps and Web Interface will not work again until this is done. 

 

After this step, service will resume; but we will need to move data from persistent disk back to a normal data disk. Another 2-3 minutes down time is expected. 

 

 

[mytaglist.com support]

This hardware upgrade is now complete, with KumoApp Engine being the last one resumed. 

[Joshué Martín]

Hi.

Today, when I stop a KumoApp, it stays n "Stopping" state forever:

[Raymond Linschoten]

Hi, I am experiencing the same problem. My KumoApps seem to be frozen up since last night, they did't react on any tag during the night. When I tried to stop them this morning they didn't respond anymore and stay in a non responsive stopping state.

Op maandag 13 februari 2017 09:14:01 UTC+1 schreef Joshué Martín:

[Slim Nil]

Same for me.  I checked this morning and my tags are responding to open/closed normally.  However not my KumoApps.  If I try and stop them and restart them all I received is “Stopping” that continues even after logging out and back into the websitetag.net site.

Sam

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-...@googlegroups.com.

[mytaglist.com support]

Are you talking about KumoApp running on my.wirelesstag.net or www.mytaglist.com?

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-tags+unsub...@googlegroups.com.
To post to this group, send email to wireless-sensor-tags@googlegroups.com.

[Joshué Martín]

In my case: mytaglist.com

To post to this group, send email to wireless-s...@googlegroups.com.

[Raymond Linschoten]

In my case also mytaglist.com. Last night my KumoApps finally stopped after ten or more hours in a non responding stopping state. I started them again which took a long time, they where running this morning but without the expected logging overnight. I believe they froze up again. It looks like I am able to start en stop the KumoApps now which takes in both cases a very long time to respond. And when they finally get started I see that the logging shows "App has been started successfully" numerous times. 

It doesn't works like it should be. Hope you can find the solution that causes these problems...

Op dinsdag 14 februari 2017 02:06:37 UTC+1 schreef mytaglist.com support:

To post to this group, send email to wireless-s...@googlegroups.com.

[mytaglist.com support]

The new server now has the tag name audio and pictures migrated. 

[Alexey Popov]

Looks like my.wirelesstag.net is down. Getting The service is unavailable.

[Slim Nil]

All my tags arm and still fire when there is movement. No server issues at this time. Perhaps there is staged maintenance going on though.

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-...@googlegroups.com.

[steve]

all my tag amangers are down at 3 difrent locations

[Chris b]

Yeah all 3  of mine are down as well.

Called support was on hold for a bit then went to voice mail where I left a message.

Not sure why they do not at least post a message or issue a email warning.

The support from this company has always been pretty bad.  One man show I suspect.  ?

A few changes and this company would be much better.  

They could start with having a a twitter feed for server status/news. 

[Chris b]

I should note that by down I meant I could not login or check any of them. Not sure if they would send alerts.

Able to login again.

No call back from support.

[mytaglist.com support]

my.wirelesstag.net is currently going through maintenance and should be back in a few minutes. 

[steve]

thanks for the update

[Sai Balakrishnan]

None of Kumo Apps have started running with the message that KumoAppEngine has been restarted by administrator for maintenance. When can we expect it to be back?

[mytaglist.com support]

Effective immediately, the IFTTT trigger "updated" will require you specify one tag, the "all of my tags" option has been removed due to its heavy load it places on our server. 

Please remove or update your recipe to that uses the "updated" trigger and "all of my tags" options.

We discourage use of this particular IFTTT trigger in general, to indiscriminately dump all captured temperature log data to another location.  We have now intentionally increased the latency of this trigger compared with other event-based triggers like "too hot", "too cold" to further reduce the load caused by frequent firing of this trigger, to ensure that our server remains fast and stable for all customers. 

[mytaglist.com support]

KumoApp engine was stopped this morning in order to diagnose server performance issue. Now we have found the culprit being IFTTT updated trigger, we have restarted KumoApp engine since 1 hour ago. 

[mytaglist.com support]

The older server, mytaglist.com will go through a max 5 minute maintenance Saturday night (11PM PDT). Since Backspace is going to put one of our disks offline for 1 hour due to their maintenance on May 1st, we decided to replace the disk before that happens, so we can control and minimize the down time. We will have to stop the database, copy about 50GB of data from one disk to another, then restart the database. 

[mytaglist.com support]

This maintenance has been completed successfully last night. 

[Advansys]

Great, thanks for the update.

[Sai Balakrishnan]

Is there an upgrade on going? I see my Kumo apps have the following message "KumoEngine has been restarted by  administrator for maintenance. Your app should restart automatically...". The apps aren't running yet.

[Simon Derbyshire]

I see the same issue, all my apps and not running because of this error.

[steve]

I'm having the sameissue with my kumoapps.

"KumoEngine has been restarted by  administrator for maintenance. Your app should restart automatically

But I recived no alert there was an issue. as some customer, i.e.use Kumoapps for thier home security cover this means we are not covered, and as we are not alerted this can be a big issue. can you please look in to an automatic alert if this happens? So we know we are not covered.

thanks

[Simon Derbyshire]

It has just come back and is now working for me.

[mytaglist.com support]

This evening, my.wirelesstag.net and mytaglist.com will have to restart in order to install security patches; the down time should be no longer than 5 minutes. 

[mytaglist.com support]

We have completed the security patch install and reboot.  

[mytaglist.com support]

We are performing maintenance at my.wirelesstag.net, during this time, historical temperature/humidity log data from December 2016 to 12/18/2017 will not be available.  The maintenance should not take more than 2 hours. 

[mytaglist.com support]

This maintenance has just been completed for my.wirelesstag.net, the log data access has been restored. 

 We will perform the same maintenance on mytaglist.com starting now. 

[mytaglist.com support]

This has been completed for both mytaglist.com and my.wirelesstag.net. 

We periodically partition the temperature log table to ensure very fast graph load performance even as the number of records grows exponentially. 

[mytaglist.com support]

The rackspace server hosting KumoApp engine for customers on mytaglist.com is currently down due to Rackspace issue, see below.  

Rackspace

Jan 9, 2018 - 3:15 PM PST

This message is to inform you that the host your cloud server 'kumoapp' resides on alerted our monitoring systems at 23:15 UTC. We are currently investigating the issue and will update you as soon as we have additional information regarding what is causing the alert.  Please do not access or modify 'kumoapp' during this process.

Please reference this incident ID if you need to contact support: CSHD-1AG241YRPQz

If you have any questions, please contact a member of our support team by visiting us in live-chat at https://mycloud.rackspace.com/, by calling 1-800-961-4454, or INTL +44-20-8734-4345.

Best Regards,
The Rackspace Cloud
US Toll Free: 1-800-961-4454
International: +44-20-8734-4345

[mytaglist.com support]

If your tag manager is hosted by mytaglist.com and has orange and red arrow lit, please power cycle it and it will reconnect.  Below is response from Rackspace rep. We will create find another server to host the KumoApp. 

Me: my server is currently down according to this ticket, can you let me know if this is a problem with the application running on my server or something on your side?

Gerard J: That is correct.

Gerard J: This is an issue with the hypervisor that the server resides on.

Gerard J: We are currently actively working on getting this back online.

Me: Do you recommend that I terminate this instance and create another instance to avoid this in future? The same thing has just happened just 2 weeks ago

Me: See https://mycloud.rackspace.com/cloud/748653/tickets#171226-ord-0000138

Me: I just happened on Jan 1st

Me: Now again

Gerard J: If you would like to get ahead of this, you can have a second instance.

Gerard J: but normally if we run into this issue we normally have a fix fairly quickly

Me: Does this problem only affect a percentage of server on ORD? Is DFW less likely to be affected?

Gerard J: Unfortunately it is a hardware issue. There is no way to track when something like this would happen or how long exactly it would take to get back online

[mytaglist.com support]

FYI Rackspace fixed the hypervisor problem and the server for KumoApp engine was automatically rebooted about 20 minutes after the outage started.  Fortunately this instability in Rackspace only affected the KumoApp engine server not the server hosting the core features. 

[Jex Webster]

FTLOG, I thought you were bright enough to avoid MS Windows !?!

[mytaglist.com support]

We have recently made a change in the KumoApp engine to improve reliability. 

The event handlers registered inside another event handler is no longer going to work.  You must register event handlers at the start of the app, i.e. in the top level of the app, and outside of any event handler. 

During the reboot of the KumoApp engine, some KumoApps that are intended to be executed only once may have been automatically executed by mistake.  Some KumoApps that you have enabled recently but did not trigger may have been turned off by mistake.  We apologize for any inconvenience this may have caused. 

[mytaglist.com support]

Between 9/20/2018 5:30-6:30AM UTC, we will perform scheduled maintenance at my.wirelesstag.net to apply a patch to the database. There will be a maximum 10 minute down time as we reboot the server and install update. 

[Christopher Mihelich]

Service down right now?  I'm getting this message "The service is unavailable." for https://wirelesstag.net/eth/

[Wireless Sensor Tag Support]

Today 5/29 between 8PM-9PM PDT (4-5AM UTC 5/30) we will reboot to install updates on my.wirelesstag.net.  This server will be unavailable for 5-10 minutes.  Same thing will happen on 5/30 between 8PM-9PM PDT for mytaglist.com. 

[Zhiheng Cao]

I am not getting it. 

--
You received this message because you are subscribed to the Google Groups "Wireless Sensor Tags" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wireless-sensor-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wireless-sensor-tags/6b85fbfb-581d-4721-b3f9-6fa201bf788e%40googlegroups.com.

[caoga...@gmail.com]

Today 9/3 between 11PM to 11:30PM PDT we will reboot the servers (my.wirelesstag.net first, then mytaglist.com)  in order to update software. The purpose of this update is to allow 3rd party modified webUI SDK to continue to function under recently updated Chrome browsers that require SameSite cookie attribute for cross domain API calls.