Next Long Shutdown

 

 

Possible Accelerator Maintenance

There is a magnet in the accumulator with a turn to turn short, which is resulting in a  1% impact on pbar stacking rate.   The spare is under test.  If the magnet fails, it is expected to take from 6 to 8 days to replace, during which time no stacking will be possible.  (A store could be dragged out, and Tevatron studies might still be possible during that time.)

 

There are also issues with the Pelletron discharging.  The Pelletron experts are looking for an opportunity to access the Main Injector for servicing, which would require draining the SF6.

 

The AD Run Coordinator does not currently consider either of these issues to require urgent response, however, we should be prepared to take best advantage of a possible few day access opportunity should it become potentially available.

 

Some candidate activities:

·       Installation of grounding screen for calorimeter cable trays on platform

·       Installation and commissioning of R2D0

·       ICD high voltage optimization

·       ICD individual channel recoveries

·       Further stabilization of the low beta quad girders

·       Checking routing of cables in racks (such as M119 and M120)

·       Documentation

 

 

 

 

Schedule Information

 

The Fermilab program planning schedule is available via this link. The Accelerator Division Operations Department has expressed interest in a several week shutdown during the summer of 2010 to perform preventative maintenance on the electrical distribution systems.   There is no clear official plan for such a shutdown at this time.

 

All requests for survey and/or alignment crew support during the shutdown should have been submitted through Bill Cooper (Cooper@fnal.gov) with copies to d0rc@fnal.gov.

 

Candidate Task List for Next Long Shutdown

      

          Infrastructure

·       Perform surveys/alignments

·       UPS maintenance—Cryo and HVAC

·       Improve radiation shielding at low beta quads?

·       Improve stability of low beta quads

·       SNEG bake-out?

·       Routine maintenance

o      Check transfer line vacuums and pump as appropriate

o      Warm up solenoid lead

o      Warm up VLPC cryostats to 50K

o      Warm up west VLPC lid gasket

o      Replace ODH heads in collision hall

o      Check Freon charge on silicon chillers

·       Investigate possible water leak in east cathedral between SMT power supplies?

·       Perform safety system tests

o      Test remote power down buttons

o      Check toroid bus connections

o      Check torques on solenoid bus connections

o      Test rack protection systems?

·       Crate/Rack maintenance

o      Verify blanks in open slots of crates?

o      Clean heat exchangers as appropriate

o      Check status of fans?

o      Check status of Cal preamp fans

·       Volk’s water level work again

 

LUMI Monitors

·       Replace luminosity monitor scintillators?

             

          Silicon Microstrip Tracker

·       Complete R2D0 installation and commissioning

·       Check silicon chiller setpoint calibration and adjust as appropriate

·       Investigate possibility of reducing chiller operating setpoint?

·       Investigate possibility of increasing humidity?

·       Replace suspect Interface Boards

·       HDI recovery effort

o      HDIs that failed as gap was closed last time

o      Download failures

o      Investigate currently disabled chips

·       Complete watchdog installation and commissioning

·       Address LVPS readback issues

 

          Central Fiber Tracker

·       Thermal cycle of cryostat—reset heater operating points to 9K

·       Investigate possibility of external crash button for AFE power

·       Evaluate and deploy readout improvement (to save 200 nsec per L1 accept)

·       Complete Temperature Control fix implementation?

·       VRB re-assignments to balance loads?

·       Testing AFE spares on the platform

·       Hunt for sources of rare coherent discriminator noise

·       Check on the power distribution for the AFE fans?

·       Investigate re-biasing of VLPCs?

 

          CALorimeter

·       Top off liquid argon levels?

·       Address cal preamp power supply failures if any

·       Continue investigating root cause of HV related purple haze

·       Check CAL HV supplies calibration

·       Address individual channel issues

o      Isolate sparking channels

·       Investigate ICD channels that are still not performing up to expectations

o      Individual channel recoveries

o      PMT replacements

o      ICD preamp connector update

o      HV stabilization

o      HV adjustments

·       Improve shielding of cable paths against muon noise

 

Central MUON

·       Remove two PDT wires and inspect?

·       Complete the process of cleaning up 1553 communications paths?

·       Test control boards in attempt to improve hot link reliability

·       Verify thin FEBs in all locations in upper cathedral

·       Recover failed A Layer PDT if any

·       Clean PDT HV service cards as appropriate

·       Relocate PDT FEBs to optimize location of FEBs with ATMEL chips?

·       Recover disabled A layer scintillator channels if any

 

Forward MUON

·       Check all Front-Ends and refresh MDT firmware

·       Verify calibrations

 

Level 1 Central Track Trigger

·       DFEF investigations and revival?

·       Eliminate master slave configuration of power supplies (if not previously addressed)

 

 

Level 1 CALorimeter

·       Hunt for individual cell noise sources

·       Investigate and recover disabled trigger towers (if any)

·       Run VME I/O stress test

 

Level 1 Muon

·       Verify spare status

·       Address power supply readback issues

·       Perhaps replace onboard batteries?

 

Level 2

·       Investigate source of global mismatches—has not been occurring recently

·       Investigate source of trigger rate dips

·       Investigate cause of timeouts on run starts

·       Enhance tools to identify source of FEBs

 

Level 2 Silicon Track Trigger

·       Complete investigation of the performance problems when beam is near 0

·       Verify error handling and busy assertions?

 

Trigger Framework

·       TFW maintenance and stress tests

·       Investigate M122 top scalar crate hangs

·       TCC maintenance/upgrades?

·       Trigger rate tests?

 

          DAQ/Online

·       Upgrades/Maintenance as appropriate

·       Test “new” SBCs

·       Test DAQ functionality with network link disconnected (~2 hour test)

 

 

 

Shift Scheduling During the Shutdown

 

Supervised access keys are to be issued by only Captains, Ops shifters, or the Run Coordination team.

If a supervised access key is required after 1600, or before 0800, the Ops shifter will issue the key.

The control room shifter is asked to verify that the issued keys are understood at the beginning and end of each of their shifts.

 

 

Access Plan

 

Supervised access keys are to be issued by only Captains, Ops shifters, or the Run Coordination team.  Please make sure to return your key before leaving the area.

 

Supervised Access Procedures

 

Don’t forget to frisk yourself and survey anything that comes from the collision hall as you exit.

 

Gap Access Procedures  

Only those qualified for gap access are allowed in the gap.  Pocket dosimeters must be worn while working in the gap.

 

 

Additional Reference Information

 

       DAQ tutorials

                   Running the DAQ

COOR

                   Data Flow

 

 

       2009 shutdown web page   

 

2008 shutdown web page

 

2007 shutdown web page

 

          2006 shutdown web page

 

       Fermilab Environment, Safety and Health Section

 

          PPD Implementation of Integrated Safety Management Policy, Hazard Analysis Procedure and Form

          PPD ES&H Tools

 

 

Last updated: 18 October 2009    G. Ginther  numeric page 1-630-218-3217