HPF detector issue and upcoming planned downtime

[Steven Janowiecki]

Dear PIs with active HPF programs on HET,

I'm writing to inform you about a recently identified problem with the HPF detector that has been growing in significance recently. In short, the bias/pedestal level of the detector has been creeping upwards and is now at about 38k counts (saturation is at 65k). This problem has reduced its effective full well capacity and observations of very bright stars may have saturated the Analog-to-Digital converter in the last ramps of the exposure.

The HPF team has a plan to address any observations of bright stars in their data reduction and Goldilocks is adjusting to handle the new bias level and some re-reductions may be necessary for the affected data. In the end we expect the impact will be a small reduction in the SNRs of bright star observations (since the last few frames did not actually detect any additional photons, or at least, none that got converted into electrons).

I've already emailed all programs with targets requesting SNGOAL>450 to ask that those observations be deferred. We believe that saturation is likely around SN=550 at this time, and discourage any observations that exceed SN=450 until the issue is resolved.

Going forward we are planning some HPF engineering downtime the week of Nov 1-7 to investigate and attempt to mitigate this problem. It is unclear whether one week will be enough time to address it, so we may also need to remain down for the following week or subsequent times to be determined. We'll keep everyone informed as the investigation and remediation processes continue.

Please let us know if you have any questions about observations you have obtained on bright stars recently or about plans you may have for upcoming observations of bright stars. We're happy to do what we can to help in this process.

Steven

[Steven Janowiecki]

Dear HPF PIs with active HET observing programs,

I'm writing to share a brief update on the ongoing HPF detector maintenance underway this past week. As you know, we paused all HPF science observations on Monday Nov 1st when this work began. During the past week the HPF instrument team has made progress, but HPF is likely to remain offline for a few more days. See below for an update from Chad Bender:

------

We have corrected the bias level and saturation problem, but do not understand the root cause of it and so expect that the bias level will continue to drift upwards (necessitating continued corrections at later times). We have also discovered that the linearity maps that were being used previously are no longer applicable and need to be rebuilt. Doing so is challenging - the original maps were created at PSU in a test dewar that allowed us to illuminate the full H2RG evenly, which is not possible when it is installed in the spectrometer.

-----

We will do our best to expeditiously complete the required maintenance and engineering tasks necessary to get HPF back online for science. I will keep everyone informed as we make progress. Please let me know if you have any questions or concerns,

Steven

[Steven Janowiecki]

Dear HPF PIs with active HET observing programs,

The HPF instrument team has released HPF for science observations tonight, after adjusting the bias level and taking some new detector calibration data. They believe that this adjustment has bought us time (at least 4-6 months) for continued science operations before the bias level becomes an issue again. Since the underlying cause is still unknown you should be warned that there may be unanticipated problems, but we are doing our best effort to get the highest possible quality data.

Please be aware of the limits of saturation for HPF! We are currently restricting targets to be no brighter than SNR=550 (that would be Jmag=4 with a 60s exposure    or   Jmag=7.2 with a 945s exposure).

If you have "Active" targets in your program, these will be observed starting tonight, weather-permitting!

Please reach out if you have any questions or concerns and we'll do our best to help,

Steven