[OT] Generator testing

[Bruce Dawson]

Dan Jenkins wrote:
> This is only tangentially related to Linux, but I'd like to gather your 
> thoughts.
>
> We have a client who installed a backup generator for their buildings 
> some months ago, which is all to the good.
>
> The unusual thing, to me, about it is that they test it in the middle of 
> every week during normal work hours. First thing in the morning, the 
> power in all the buildings goes off for about 20-60 seconds when they 
> switch to the generator, and then does it again 5-10 minutes later when 
> they switch back. The servers, all but one running Linux (to bring this 
> slightly on-topic), are protected by UPSes, so they aren't directly 
> affected. The network switches, wireless access points, printers, and 
> computers scattered throughout the buildings, however, are not so 
> protected. Everyone is supposed to turn off their equipment Tuesday 
> night, but often individuals forget. They have noticed it takes the 
> network awhile to operate normally after this weekly event. Not 
> surprisingly with a flood of DHCP requests and SMB network convergence 
> happening when several hundred computers are turned on. Some of the 
> network switches and access points require a power cycle. We have seen a 
> slight increase in equipment failures, largely among the oldest devices. 
> We have proposed a plethora of UPSes to support the network devices, 
> however, that has not yet been approved, and may not be until next year. 
> We have also requested a less frequent test cycle and relocation to a 
> weekend, so far to no avail.
>
> My question, is it normal to test a generator weekly, during operating 
> hours, and is it normal to shutdown all the power in the buildings to 
> test it? It seems wrong to me & to the maintenance director, however, 
> the company which installed the equipment says this is normal operation. 
> Someone involved with the project implied it is a legal requirement in 
> some manner. Does any of that jibe with others experiences?
>
> Thanks enormously for any feedback.
>   
Its more a function of their ATS (Automatic Transfer Switch). For
example, our ATS (for the 50KW generator we have here on the farm) has a
weekly clock (that does not understand daylight savings time, sigh) to
perform the automatic exercising.

But, be glad that they are dumping the power for 20-60 seconds during
the "exercise interval"! It may be inconvenient, but if the power was
not dumped, there will be a "phase change" during the switchover that
will [eventually] destroy all the motors that are on the generator
circuit. (For motors, think: freezers, fans,
pumps, ...) I've also heard that those phase changes are hell on solid
state power supplies, but don't have any solid direct evidence.

If they had synchros on the generators, then that would make the
transfer nearly seamless. However, you're talking about a much more
expensive genset. Most gensets smaller than 100KW don't have phase sync
abilities, and those that do require additional expensive equipment to
take advantage of them.

BTW, I haven't found a line-interactive UPS that will protect against
generator initiated phase changes. Just dumping the power for a few
seconds is the cheapest most practical approach to protecting equipment
against phase shifts.

However, I did find something odd about what you said... Evidently the
test period is only "5-10 minutes". I would be a little concerned that
the genset has had time to warm up in that period. We've set our test
period to be 1 hour and during "prime time" to put as much of a load on
the genset as possible. (Most gensets hate variable loads, and the best
way to keep them "nimble" is to exercise them with a big load.)

Another thing they should be aware of is that the fuel tank is heated.
We've had ice crystals form in the fuel filter, thus clogging it, and
preventing the genset from properly activating during a real power fail
(or exercises). This was a real problem to diagnose because by the time
we got it running, the ice had melted and the generator would activate
flawlessly. I never would have figured out the problem unless something
similar happened on one of our tractors and the tractor repair shop told
me to check on this.

I've got lots of practical experience (and horror stories) if you're
interested in hearing about them off-list.

--Bruce