Water level data extraction

[Vikas Rana]

Is there any way by which we can extract the data for all the dates from https://vmc.gov.in/waterlevel.aspx website? Actually, I want  28 July 2019 to 2 august 2019 data for a 2-dimensional flood simulation, but the extraction of all the data would be much more beneficial. Thank you.

[Palash Kulshrestha]

i dont think there is any option to retrieve old data. i can help you to scrape if you know how .

[Palash Kulshrestha]

in wayback machine there is data for 12 august

https://web.archive.org/web/20190812194137/https://vmc.gov.in/WaterLevel.aspx

[Vikas Rana]

I don't know how to scrape the data... Any guidance would be helpful.

[Dilawar Singh]

Web-scraping requires knowledge of some programming language. In essence, it is an art of transforming HTML to a format you like. For common tasks, you can find some tools easily on the net e.g., extract all tables from this URL and save them as csv.

For example, you can get the html from the above link using `curl` and pass it to `pandoc` (or html2text) to convert to plain text.

[dilawars@localhost Downloads]$ curl https://vmc.gov.in/WaterLevel.aspx | pandoc -f html -t plain
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 14330  100 14330    0     0  33325      0 --:--:-- --:--:-- --:--:-- 33325
Water Level Statistics

Ajwa

Vishwamitri

      Date      Time    Level (Feet)
  ------------ ------- --------------
   17/06/2020   06:00      207.65
   17/06/2020   03:00      207.55
   17/06/2020   00:00      207.55

      Date      Time    Level (Feet)
  ------------ ------- --------------
   17/06/2020   06:00       4.00
   16/06/2020   21:00       4.00
   16/06/2020   18:00       4.00

Copy paste this in a Libreoffice-Calc or MS OFFICE spreadsheet and see if it gets converted to a spreadsheet.

Anyway, this webpage does accept a POST request but only in encrypted format. So it is essentially useless. Contact the author of https://craigdsouza.github.io/data/IMD-Hourly-Precipitation-Data, he scraped rainfall data from IMD website (2018). This data is probably of no use to you but he might know a better source to get this data. I couldn't locate time series data for a city on IMD website.

best,

   Dilawar

On Tue, Jun 16, 2020 at 10:42 PM Vikas Rana <vikni...@gmail.com> wrote:

I don't know how to scrape the data... Any guidance would be helpful.

--
Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/a5f65448-059b-4132-828c-ff36ccc60310o%40googlegroups.com.

--

Dilawar
NCBS Bangalore

[Palash Kulshrestha]

 i dont think we can get old data from this website, at least from current information.

On Thursday, June 11, 2020 at 5:25:38 PM UTC+5:30, Vikas Rana wrote: