Richmond releases drafted after-action report on water crisis

RICHMOND, Va. (WRIC) -- The City of Richmond has released the draft of the after-action report conducted by a third-party firm on the January water crisis.

The report, conducted by HNTB Corporation, includes more detail on the events that occurred on Jan. 6 when a winter storm caused a power outage and subsequent malfunction at the city's water treatment plant.

The document also outlines its findings and immediate recommendations for the city.

In addition to visiting the plant, interviewing staff and acquiring documents from the city's Department of Public Utilities (DPU) as part of the report, HNTB requested an interview with former DPU Director April Bingham, who declined.

This article will be updated as 8News reviews the details of the report.

Overview of events and factors on Jan. 6

Staffing

When Richmond's water treatment plant lost power in the early hours of Jan. 6, eight staff members were working, including a plant operations supervisor, three plant operators, two regularly scheduled night shift operators, one day shift operator who had arrived early and a mechanical utility plant specialist, according to the report.

There were reportedly no electrical or instrumentation and control staff members at the plant during the power outage. Since only electrical staff are trained and able to transfer to generator power, there were no qualified staff members at the plant to switch to generator power in the event of a power outage.

Equipment

The plant was reportedly in winter mode at the time, meaning that only Main Feeder 1, the first feeder of power, was powering the facility. Main Feeder 1 went out, likely due to ice from the storm, but Main Feeder 2 still had power.

An automatic mechanism that would have automatically transferred power from Main Feeder 1 to Main Feeder 2 then failed, causing the plant to lose power completely. The Supervisory Control and Data Acquisition (SCADA) computer system went down at the same time the plant lost power.

Without power, the report said staff could not close filter effluent valves, which control water flow. The water level in both basements at the plant continued to increase, reaching over six feet.

An electrician arrived early for their shift at 6 a.m. and focused on manually transferring power from Feeder 1 to Feeder 2. The report said they did not try to start the backup generators and switch to generator power at any point during the outage.

The utility plant specialist electrical supervisor manually transferred power from Feeder 1 to Feeder 2, and the plant's power was restored an hour and 20 minutes after it was lost.

Pumps

Pumps were used to keep water out of the basements, but the rate at which they were pumping did not help due to the incoming water flow. Additional pumps were then brought in.

Water then stopped flowing through filters at 8:10 a.m., about 2 hours and 25 minutes after power was lost. It was then that the pumps began to lower water levels.

Notification of issues to DPU Director, counties

At 3 p.m., the then-DPU Director, April Bingham, was notified that the water levels at the Byrd Park Water Reservoir were low and had dropped to about 10 feet.

At 7 a.m., the superintendent at the plant reportedly notified Bingham, as well as Chesterfield County and Henrico County water plant staff, of the issues and asked that they lower their water demand. 

Hanover County, however, was not contacted until 2 p.m. -- over eight hours after the outage -- due to a lack of up-to-date contact information.

PREVIOUSLY: Richmond releases preliminary report on widespread water crisis

Reservoir had lower capacity due to construction

A construction project was underway at the Byrd Park Reservoir, Richmond's main reservoir, at the time of the water crisis. It normally has a storage capacity of about 55 million gallons.

The report said that, due to the project, half of the reservoir was out of service. The half that was in service could only be filled less than three-quarters full, which reduced its capacity to store water.

"With this constraint, Richmond's distribution system was at an increased risk to be able to meet demands in the event of loss of [water treatment plant] production," the report said.

Basement flooding was a "common occurrence"

Flooding in the basement of the plant commonly happened, according to plant staff interviewed by HNTB. That usually entailed a small amount of flooding in the lowest levels of the basement that did not damage equipment.

During the interview with plant staff, the report said that multiple staff referenced a flooding event that caused the plant to stop producing water for over six hours due to the basement flooding.

Lack of storm preparation

According to the report, despite a declaration of a state of emergency by both Gov. Glenn Youngkin and by Richmond Mayor Danny Avula due to the winter storm, there was "little proactive action" taken at the plant to prepare for a possible power outage.

"Outside of staffing the overnight shift with mechanics for snow removal, there was no formal discussion or written notice or reminders to the staff on specific actions needed to prepare the plant for a potential outage caused by the storm," the report said.

There were reportedly only informal discussions among staff during shift changes as to what actions to take during a possible power outage.

Staff only prepared by fueling vehicles and backup generators and filling chemical tanks.

Aside from backup generators being checked and noted as working on Jan. 4, the report said standby equipment was not prepared for the event of a power outage.

The report explained that "staff were not adequately prepared for the incoming storm."

Recommendations

The report includes recommendations for developing more extensive plans, increasing staffing levels during expected weather events, training staff, checking and replacing equipment and establishing protocols about the notification of counties that rely on water from Richmond's plant.