Anyone interested in working on mining data?

[Rahul Basu]

Hi friends,

I work on mining in India, primarily Goa. Over time, a lot of data has been collected through various public sources and RTI.  Much of it is in excel, a part is in GIS form. Some of the excel is old government style, needs openrefine to cleanup. 

Lots of interesting analysis can be done (for example). Some of data includes detailed information on transport of minerals from leases to jetties to ships.

I'm not very good at data manipulation / data visualization and was looking for collaborators. 

If anyone is interested, please ping me.

PS. Odisha seems to have good mining data as well, though much more data collection is needed. Can be looked at in a second phase.

Warmly

Rahul

Today is the first day of the rest of your life !

Goa Foundation | The Future We Need | Goenchi Mati Movement

https://orcid.org/0000-0002-1218-878X

The Supreme Court and Intergenerational Equity: The Goa Mining Case

[Dr. C.Gajendran]

Dear Your email find interest to me; please contact me at 9443368980. We shall discuss further.

Regards,

Dr. C. GAJENDRAN  B.E (Civil) M.E (Env.Eng.),Ph.D.,D.R.T.M.,D.I.S.,FIE(Ind.)., IGS., ISG.,

Vice Principal | Director – IQAC | Professor & Head, Civil Engineering | Vice President IIC 

Jawaharlal College of Engineering and Technology, Palakkad

Mobile & Whatsup! : +91-9443368980
skype: c_gajendran
Google Site 

ORCHID : 0000-0002-2765-7075

Vidwan

Scopus Author ID: 57359210200

Face book : 

Twitter : dr_gajendran

Instagram : dr.gajendran

--
Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/datameet/CAN8Wyx_6JXrvcYQnPSw7j%3DKZg2JEWSsB66ur4SRBGaZrsJuDRQ%40mail.gmail.com.

[Arun Ganesh]

Hi Rahul, this is a very deep topic and requires many hands to dig through (pun intended). is it possible to share links to some of the collected data so it can be explored? 

Just extracting data from PDF tables requires a lot of effort and it would be best to keep the data open so it can get maximum interest. Or are there reasons to not make this public given its sensitive nature?

To view this discussion visit https://groups.google.com/d/msgid/datameet/CAMFo2vtYsS6B%3DMs_du2eE8H3%2BYqTjcUfopNhXp0%3De_37fWqqLA%40mail.gmail.com.

[bibhu prasad mishra]

Hi Rahul,

I am very interested in collaborating on this. As I am from Odisha, which is a major mineral hub, I believe this is a great opportunity to work with this data.

I would love to help with the data manipulation and visualization, especially as you look toward the second phase involving Odisha. Please let me know how we can get started.

Thanks and Regards

Bibhu Prasad Mishra

Civil Engineer,

B.tech, M.tech.

7894761150, 9937512121

To view this discussion visit https://groups.google.com/d/msgid/datameet/CA%2BGKQr1ph9XXnZ%3Dv5qC1ikSafjUqLu9gMCJcyqE6QX_FZ5gOHg%40mail.gmail.com.

[Natasha kalra]

Hi Rahul,

Thank you for sharing this. The dataset sounds extremely valuable, especially given the combination of RTI-based information, government records, GIS layers, and transport-chain data linking leases, jetties, and shipments.

I work in the broader sustainability, environmental governance, and data-driven policy space, and I would be interested in understanding the scope of the datasets and the kinds of analyses you are looking to undertake. I am affiliated with the Institute for Social and Economic Change (ISEC), Bengaluru, a premier interdisciplinary research institute engaged in policy-oriented research, consultancy, and capacity-building across areas such as sustainability, environmental governance, urban development, and social transformation. Through our work, we collaborate with government agencies, corporates, development organizations, and communities to develop evidence-based and scalable solutions to contemporary challenges.

It would be great to connect and discuss the available data, current status of data cleaning and organization, and potential avenues for collaboration. Please let me know a convenient time for a call.

Looking forward to learning more.

Warm regards,

Natasha

Dr. Natasha Kalra

Environmental Sustainability Consultant 

Initiator- BaansCraft

https://www.instagram.com/baanscraft/

M: + 91 99167 70495

To view this discussion visit https://groups.google.com/d/msgid/datameet/CAN%3DkOZnAQb-N%3D%3DLg5kfTwCgxhG2oaorB8p3PGdcXopJnMONiFQ%40mail.gmail.com.

[Rahul Basu]

Friends,

Thanks for the interest in this, unexpected for me. I now have to sort through the data and figure out how to organise it for public use. Give me some time on this.

For the moment, here's two datasets I received under RTI around transport permits. 

Background: Goa runs an ore accounting software, which basically accounts for mineral ore at different locations. To move ore from one point to another, a transport permit is required. This is usually for a decent quantity, say 10,000 tons, to move from Mine X to Jetty Y. Then there are truck trips (usually 10 tons each), and the trucks are monitored real time. There are also trips by barge from the jetty to the ship, and some rail as well. When all the trips against the permit are completed, there's usually some difference, which gets adjusted. The software provider has changed from Megasoft to GEL. GEL software is called Bhumija.

Mining was in three major phases: pre-2012 ban; 2015-2018; and 2024 onwards. Ore is mined in Goa, most exported, some consumed in Goa, some shipped by barge to JSW Dolvi, Maharashtra (recent). Ore is also imported into Goa from overseas. Some ore mined in Karnataka and Maharashtra also moves through Goa, mostly for export but occasionally for use in steel plants in Goa. 

The first folder has transport permits. Lots of them. From 1-Apr-2020-2025

The second folder is around closing stock also has some version of the manuals for the software. I had asked for:

a) Manuals of the Bhumija software
b) Guidelines / SOPs / instruction for monitoring ore through Bhumija 

c) In excel format, for each major mineral ore transport trip after 25-Mar-2024, permit number, carrier registration number, the source location with time and weight at the source weighbridge and the destination location with time and weight 

d) In excel format, closing stock of major mineral ore as of 31-Mar-2024 and 31-Mar-2025 for all locations in Bhumija

I have a few questions from the data:

1) Does the ore balance at any location go negative at any point in time?

2) Overall statistics - major input locations, major transport routes, total quantities, etc.

3) Is the data complete? Is any further information needed / desireable?

More questions will come as other data sets get added

Please feel free to work with this while I try to set up a better data repository. Email me directly if you have questions on the data in the two folders.

Warmly

Rahul

[Sanjay Bhangar]

Hi Rahul,

Thank you so much for this most interesting dataset - was a fun thing to wake up to Sunday morning and couldn't resist spending a bit of time with AI tools to clean up the Excels a bit, ingest it into a database, and enable some basic querying and visualization. I've been wanting to play around with Datasette for some time now, and this seemed like a good use-case to get up something simply to explore the data.

 - I have put up what I came up with at https://goamines.whydidweevendothis.com/

 - List of tables, query executor and some example queries: https://goamines.whydidweevendothis.com/goamines

 - A map showing route arcs of movements: https://goamines.whydidweevendothis.com/static/routes_map.html (geolocation of points is currently imprecise)

All the code for the data ingestion, including notes made by my AI agent along the way can be found at https://github.com/batpad/goamines

WARNING: I have really not spent a lot of time verifying ingestion scripts, etc. so there could be discrepancies in the data, especially since there was quite a lot of inconsistency in the source data - please do not treat this as an accurate data source without doing your own verification.

I likely will not have a lot of time to continue working on this, but it is a really interesting data-set and it was really fun to spend some time building something with it. If folks find it useful / want to take it over / move the repo or domain somewhere etc. please just let me know and of course feel free to make Issues and Pull Requests on the repository. Also, if anyone wants the SQLite file (around 90MB) just let me know and I'll put it somewhere for download.

Thanks again Rahul - this was some really amazing work gathering this data - full credit to you, and I hope you find this experiment in processing and visualizing it useful.

Cheers,

Sanjay

To view this discussion visit https://groups.google.com/d/msgid/datameet/c67f7584-fcbc-4a1c-83ba-dff8b0e08085n%40googlegroups.com.

[Ashim Jain]

Friends, 

Just catching up on this thread.

Out of the huge datasets, if specific evidence can be extracted showing govt. and/or corporate culpability in mineral exploitation, how can we make this easy to understand and reach the affected masses in those areas?  (Because it's only the knowledge and power of the masses that has the potential to change the status quo.)

For decades, we have been reading about the tribal and rural populations' struggle against being dispossessed of the land and resources they have nurtured for thousands of years.  It is anti-democratic and further leads to concentration of wealth and power, exacerbating the wealth disparities that are causing havoc.

In most cases, it is being done illegally (manipulating the gram sabhas, under the table deals, etc.).  

I'm willing to do my part (web app development and contribution to a combined effort) if we can form a team.

- Ashim, Udaipur, Rajasthan

To view this discussion visit https://groups.google.com/d/msgid/datameet/CAG3W7ZHRAfuHmgdhRaFcAuyX4vyAHd2dZis-C46osKEqhDXy5g%40mail.gmail.com.

[Rahul Basu]

The data exists, every week there are new reports of illegal mining or illegalities in mining. 

Communication to mobilize masses is hard, but possible. Consider how there were crowds on the streets during IAC in 2012 (about corruption in coal blocks among others), which ended up with the NDA government in Goa in 2012 and nationally in 2014. 

We have a different approach which we tried through the Goenchi Mati Movement back in 2017. See the manifesto we wanted politicians to sign up to. It was very hard, we didn't succeed in changing the politics. Here's a 13 minute video of what happened. 

This is a summary of what we think would be fair and can be sustained. The key advocacy point for us has been the government as our trustee, not so much vilifying the miners as the problem is really a systemic one - everyone in the present generation is trying to capture some part of the shared inheritance of our children & future generations. Note that the target of our communication is the unaffected masses, neither benefiting from mining nor facing the ground consequences - the pitch is that their children's inheritance is being stolen and their government corrupted from the proceeds; but there's an alternative that is better without completely banning mining.

At the moment, if we find irregularities, we approach the government, and failing that, the courts. Even the Supreme Court is finding it hard to control illegal sand mining in the Chambal river. So yes, we do need to communicate better.

Warmly

Rahul

Today is the first day of the rest of your life !

You received this message because you are subscribed to a topic in the Google Groups "datameet" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/WLHGLnl5IX0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to datameet+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/datameet/CAPaFkDyWah8H8r_bw-X%3DU5myyZFYyKEVD9ohGZi3posYipD_gQ%40mail.gmail.com.