[RFC] IT Infrastructure Inventory

[Brett Neese]

Members, 

The board is currently working on inventorying, documenting and putting together a plan to upgrade some of our IT systems. 

We have identified the following critical systems to keeping the lab functional and where they are currently located:

Server rack upstairs:

• WAN/LAN - including firewall, etc.
• Wiki
• CCTV security cameras (NOT the cameras that were on the website. We have a separate system of security cams that are NOT publicly accessible and ONLY accessible to the board on-site.)
• Access control (door lock)

Offsite:

• Website
• Github
• Social media
• Slack
• Google groups
• Google workspace 

Note: other lab equipment (computers for the various stations, etc) are out of scope for this project and will be unaffected; our focus is largely on tidying up the pizza boxes upstairs.

I am reaching out for comments  - is there anything critical running, particularly on the server rack upstairs, that is not listed above and that we need to ensure to retain as part of our upgrade path? 

We currently have no immediate plans (nor intent) to shut off anything important, but we want to make sure we have an inventory to ensure that doesn't accidentally happen! So if you know of something we missed, this would be a good opportunity to mention it.

Of course, we will reach out to the membership once we have a better plan - this is just the first step in that process. 

You are also certainly welcome to direct any questions you may have about this initiative to Operations <opera...@heatsynclabs.org> or the board <bo...@heatsynclabs.org>. We welcome your input into this process.

Graciously, 

Brett 

HeatSync Secretary

PS - We know some of this work has been done before and we are thankful to all the people who have come before us! We've reviewed as much public-facing documentation as possible. 

[Antonio Contrisciani]

I dont know about how critical it is, but member file storage.

--
You received this message because you are subscribed to the Google Groups "HeatSync Labs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to heatsynclabs...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/CAKbRb0QeN%2B1y590CncNXLoEm3u0t2mn1qgNZUMryCkd9hSTB%2Bw%40mail.gmail.com.

[Brett Neese]

Ooh, that’s a good one. I’ll add it to the list.

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/CALmAu84PWH%2BKq4wmQ6bY_xAfv4K3KSdeT2w0an8oWBGh_aoSCw%40mail.gmail.com.

[David Lang]

Brett Neese wrote:

> *Note:* other lab equipment (computers for the various stations, etc) are

> out of scope for this project and will be unaffected; our focus is largely
> on tidying up the pizza boxes upstairs.
>

> I am reaching out for comments - *is there anything critical running,
> particularly on the server rack upstairs, that is not listed above *and

> that we need to ensure to retain as part of our upgrade path?
>
> We currently have no immediate plans (nor intent) to shut off anything
> important, but we want to make sure we have an inventory to ensure that
> doesn't accidentally happen! So if you know of something we missed, this
> would be a good opportunity to mention it.

I think we can make a broader statement, we are looking to figure out what's
running so we can try to avoid interrupting anything. If there is stuff running
that isn't considered 'important' we still don't want to shut it off because we
didn't know about it. If anything gets turned off, we want it to be a deliberate
decision (with the general default being to keep things running until we are
happy with it's replacement), not because the right person didn't know that it's
there.

David Lang

[Darrell Wertz]

The door locks are on the system. I hope you have the time and money to get things done. You may want to check with Nate Caine.

Darrell Wertz

717-578-9884 Mobile

--
You received this message because you are subscribed to the Google Groups "HeatSync Labs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to heatsynclabs...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/r4701p6r-8qo3-2r95-q4pr-s936o4o6r835%40ynat.uz.

[Luis Montes]

We have stuff in the server rack that we don't understand, so we need to know what the bare minimum is to keep the lab functional.  We really should shut down and remove old projects that we don't have anyone supporting.

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/CAJRzTqRDuY0Ux16j8rRwn6M_CJPLbARdZrE_J0CwOitUY0Pa3g%40mail.gmail.com.

[Paul Hickey]

Whats our budget outlook? 

With our current deficits, how long do we foresee the lab being open and when is the go/no go for deciding to close the lab if we can't make up the difference between our operating costs and our income? 6 months? 12 months?

Seems like we'd be rearranging the deck chairs on the Titanic without that info.

I'd love to hear some great news that we are fine and we have a small monthly surplus to do improvements like this.

Paul Hickey

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/CA%2B%3DVRdDVFSb3rcN7q2HoBOWktoPZ9Pi6%2Bmh%2Berq6%3D1nQd84m9w%40mail.gmail.com.

[David Lang]

On Mon, 3 Nov 2025, Paul Hickey wrote:

> Whats our budget outlook?
>
> With our current deficits, how long do we foresee the lab being open and
> when is the go/no go for deciding to close the lab if we can't make up the
> difference between our operating costs and our income? 6 months? 12 months?
>
> Seems like we'd be rearranging the deck chairs on the Titanic without that
> info.
>
> I'd love to hear some great news that we are fine and we have a small
> monthly surplus to do improvements like this.

That is not the case (unfortunantly)
That said, there are some things we can do that, if they had been done a year ago, would have tipped us over the balance last year.
This obviously means that the rumors that we are going to run out of money in the next few months are wrong

The board is still working on nailing down details. But I don't want to go into partial detail while we are still figuring out projected expenses. We hope to get a report out at the next HYH but have promised to get details out this quarter.


This IT Infrastructure project is not focused on new hardware (although it would not be hard to get new hardware donated that is better than what we have been running for a long time) as much as it is getting software up to date and supported.

There is some software that is horridly out of date, and there is custom software that nobody has been maintaining and nobody currently knows how it all works. We want to get everything updated to current versions, but we don't want to waste a lot of time working to reverse engineer things that are running as experiments that nobody cares about. But we don't want to take the approach of "shut things down and see who screams" :-)

Thus the questions about what is in use.

David Lang

[Brett Neese]

Should be noted that shutting off unnecessary/unused hardware and migrating things to places that are cheaper to run (ie - Azure gives us $2K/year, many of our services will easily run within free tiers of other providers) is one of those things that would help reduce monthly expenditures. 

Brett

--
You received this message because you are subscribed to the Google Groups "HeatSync Labs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to heatsynclabs...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/ps37n543-9r04-1ro4-1883-0129rnosor68%40ynat.uz.

[David Lang]

Brett Neese wrote:

> Should be noted that shutting off unnecessary/unused hardware and migrating
> things to places that are cheaper to run (ie - Azure gives us $2K/year,
> many of our services will easily run within free tiers of other providers)
> is one of those things that would help reduce monthly expenditures.

newer hardware with SSDs is going to be faster and consume less power than older hardware with spinning disks for those things that need to run on-site (doors/firewall/etc)

But worrying about a couple pizza boxes while leaving all the other computers running seems like we're ignoring the majority of the power consumption :-) but we are working to get better power consumption data to see how much it matters.

David Lang

[Darrell Wertz]

Do we currently pay $2k a year for Azure?

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/CALOuO4N39i5gpnmN%3DTRR9mDRGmCBcZTx-%2BiM%3DFLSDZYpuqoCpw%40mail.gmail.com.

--

Darrell Wertz

717-578-9884 Mobile

[David Lang]

Darrell Wertz wrote:

> Do we currently pay $2k a year for Azure?

I think it's that Azure will donate $2k of services to us each year.

David Lang

>> https://groups.google.com/d/msgid/heatsynclabs/CALOuO4N39i5gpnmN%3DTRR9mDRGmCBcZTx-%2BiM%3DFLSDZYpuqoCpw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/heatsynclabs/CALOuO4N39i5gpnmN%3DTRR9mDRGmCBcZTx-%2BiM%3DFLSDZYpuqoCpw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>
>
>

[Brett Neese]

Correct David - 

https://www.microsoft.com/en-us/nonprofits/azure

And they’re certainly not the only provider who can give us for free what we’re currently paying (at least electricity) for. 

That being said, the most obvious and immediate win is shutting down anything we’re currently paying for and not using.

 For example, we don’t know what’s running on the box labeled virt0 and there seems to be universal agreement that nobody does and we should just unplug it. (I excluded this detail from my original post because I wanted to see if anyone mentioned it, so far it doesn’t seem like it.)

But, we don’t want to move anyone’s cheese, we have no intent to disrupt things that ARE in use. Hence the request for input on our inventory.

Brett

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/7pqqr6ns-20s0-q0os-p034-937n21104797%40ynat.uz.

[Darrell Wertz]

We don't want our data sequestered on someone else's cloud. Nor do we want to get roped into paying exorbitant fees when the subscription model changes.

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/CALOuO4MHdA06d3UMSwgNDTTQ8FrHVeNRhidkuowBPU8wt-JBzw%40mail.gmail.com.

[David Lang]

Darrell Wertz wrote:

> We don't want our data sequestered on someone else's cloud.

We absolutly need to own our data, but that can be done in someone else's cloud
(there a lot of businesses with FAR more on the line than us that have their
data in the cloud

> Nor do we want to get roped into paying exorbitant fees when the subscription
> model changes.

This is a bigger issue for me. It's nice that companies are doing well enought
to be hadnign out services without charge, but they need to beneift from it or
it will eventually stop (it may be that the 'first taste is free' attraction to
get people to buy more of their service is good enough)

so while I am willing to consider running things in the cloud, I get very
nervous about building systems around features only available in one vendor's
cloud.

For our scale, I want to have a copy of things in the cloud, as a backup and as
a fallback if the building looses connectivity or power, but I want to have it
built so that it can exist in multiple places, and for our scale, a local
instance works very well.

>> https://groups.google.com/d/msgid/heatsynclabs/CALOuO4MHdA06d3UMSwgNDTTQ8FrHVeNRhidkuowBPU8wt-JBzw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/heatsynclabs/CALOuO4MHdA06d3UMSwgNDTTQ8FrHVeNRhidkuowBPU8wt-JBzw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>
>

[Brett Neese]

Maybe, but that's not what this thread is about and no one is proposing any immediate changes other than decommissioning servers that nobody seems to have access to and that do not appear to be doing anything. My immediate point is  if we can take hardware that is not even being utilized out of commission that will have a net positive effect on our balance sheet. Keeping this stuff running is also an existential risk for our organization in terms of liability, which is why we're doing this work now.

Currently, we're at:

Server rack upstairs:

• WAN/LAN - including firewall, etc.
• Wiki
• CCTV security cameras (NOT the cameras that were on the website. We have a separate system of security cams that are NOT publicly accessible and ONLY accessible to the board on-site.)
• Access control (door lock)

• File server

Offsite:

• Website
• Github
• Social media
• Slack
• Google groups
• Google workspace 

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/CAM6Z%2BcewMWtxQ0zTdrrED2ZDLH%2BbNwzK%2BCfcUmjykt%3DsLaognQ%40mail.gmail.com.

[Luis Montes]

We have our google workspace and this mail group in the cloud.  Slack is in the cloud.  Our code and public website is on github which is in the cloud.

Would be nice to be able to pull regular backups of data, but I don't think we can keep everything local if we wanted to.

To view this discussion visit https://groups.google.com/d/msgid/heatsynclabs/99044s47-612p-8o14-o27s-pqo105rqs229%40ynat.uz.

[Cprossu]

To elaborate a little bit on the WAN/LAN
Our WAN/Gateway: There are 2 Arris Cable Modems (DOCSIS 3.1 CM8200/SB8200), one is in use, one is a backup 

Our Firewall: There are 2 SuperMicro servers with 4x10gbe ethernet adapters built in. I have removed 1 cpu and removed most of the ram to make it more energy efficient, but the other is not configured that way. The online machine is running PFSense and 2 Samsung sata 870 evo 500gb SSD's mirrored for storage. We need 2 more SSD's in order to make the other server into a swappable spare.

Our LAN: There's a Ubiquiti switch up there running everything and a Cisco gbit switch as a spare, we are running recently updated Ubiquiti AP's for the WIFI, and also a Ubiquiti Cloud Key running the Ubiquiti OS to run the configurations for the switch and the AP's

Our Door Access Control: There are 2 medical grade DC battery backups which use LiFe cells for power, one powering the door control box along with a 24V Boost power supply for the maglocks. The second unit is a working spare 

Our member DB and Wiki: currently run from one of the servers on the rack, I am not sure if we are on new hardware or still running the ancient hardware, but the goal was to take everything off the old dell server running it prior and push it to an updated machine.