Instrument Log

 

WACR C1 CF Lamont, OK

 

Date

Technician

Comments

9/23/08

CAM

The WACR needs to be removed and sent to the vendor to be upgraded to a SWACR (scanning WACR). Powered the WACR down at 1813 GMT. Removed *RF unit, **computer, ***scope, ****PDU, *****Lacie Drive, ******modulator P/S and all cables requested by vendor to be returned for upgrades. System is offline. *Old - US001, **Old - US001, ***Old - C040082, ****Old - ZA0427000354 *****Old - 162504539 ******Old - 01

9/8/08

PAD

Instrument and computer have power but program is not running. Tried to start with insect d, but it status shows "cannot start, insect d appears to be running with PID 5977. Program is not responsive to insectl -s or insectl -k. Shut down/ restarted computer and started program but when I tried to bring up monitor graphs, error message read: Program Insect received and X Window System Error. Shut down with insectl -s and restarted system and program. Instrument and monitor graphs are now running correctly. Thunderstorms in area at time of outage. RTS at 1520 GMT.

8/25/08

PAD

Last SDS collection was on 8/23/08 at 0110 GMT. Instrument seems to be operating but no graphs are shown. End of file error encountered on graphs. Cleared error message. GUI graph has an Error Reading Reply. Will not connect with computer on GUI window to start recording. Tried insectl -s to stop program and instrument stopped radiating but instrument rack did not power down as normal. Vendor called at 1420 GMT and said he had remotely started instrument approximately 45 min earlier. After stopping/ restarting program was I able to connect with GUI to start recording and bring up monitor graphs. RTS at 1446 GMT.

8/19/08

PAD

Last SDS collection was on 8/18/08 at 0717 GMT. External data disk use percentage has not changed since yesterday. Found instrument operating and graphs are shown. Tried to contact mentor and vendor to find out if data is still being written to external data disk. Vendor returned call and advised to start data collection as normal. Stopped/ restarted program and started recording data. RTS at 1521 GMT.

8/4/08

PAD

R/R external data disk at 1551 GMT. RTS at 1611 GMT.

7/14/08

MK

CF lost power on 7/13/08 between 0028 and 0137 GMT. The WACR is not currently running. Restarted WACR computer and software at 1450 GMT.

7/9/08

PAD

CF experienced power loss from 0745 to 1358 GMT. Computer has power but is not running. Powered up instrument and started recording data. RTS at 1545 GMT.

6/16/08

PAD

Power outages at 0429 GMT and again at 1834 GMT. Computer and instrument went down after both outages. Restarted computer and instrument at 1431 GMT and again at 2023 GMT. RTS at 2043 GMT.

6/9/08

PAD

CF experienced extended power outage early on 6/9/08. Computer has power but instrument is not running. Restarted program at 1514 GMT. After letting instrument warm up, I came back to start program and error message was shown: Modulator faulted while warming up. Modulator deck fault. Restarted instrument and started recording data at 1552 GMT.

5/29/08

PAD

Last SDS collection was on 5/28/08 at 1510 GMT. Mark Klassen R/R in rack UPS battery this morning. I checked the instrument at 1510 but could not access external data disk. Stopped program and stopped/ restarted external disk. Restarted program. RTS at 1530 GMT.

5/28/08

PAD

Last SDS collection was at 1710 GMT. UPS and computer have power but systems are not running. Restarted program and started collecting data. RTS at 1353 GMT.

5/27/08

PAD

Last SDS collection was at 0910 GMT. UPS and computer has power but system is not running. Started system and let it warm up. Started recording data at 1430 GMT.

5/1/08

PAD

R/R external data disk at 1520 GMT. RTS at 1540 GMT.

4/28/08

PAD

Last SDS collection was on 4/26/08 at 0010 GMT. CF experienced extended power loss approximately 0000 GMT. Instrument rack has power but computer is not on. Started computer and program. RTS at 1627 GMT.

4/8/08

PAD

Power outage at CF between 0709 and 0910 GMT. Instrument control rack has power but computer is off. Started computer and instrument. RTS at 1643 GMT.

3/31/08

PAD

Found instrument shut down. Log shows : 0729 Software overflow. 0742 Cannot write spectrum Input/ Output error. Notified instrument mentor and restarted instrument. RTS at 1627 GMT.

3/19/08

Mentor

Mentor noticed WACR was in co-pol only mode. Mentor remotely changed to cross-pol mode at 1530 GMT.

3/7/08

PAD

External data disk has reached 92% capacity and needs to be changed. At 1529 GMT, stopped WACR and R/R external data disk. WACR restarted correctly. RTS at 1544 GMT.

3/5/08

Mentor

Blower found inop again. Suspect motor has reached the end of its useful life. Mentor and vendor on site and chose to disconnect the blower power and run system without it from now on. WACR will be removed from service soon to be upgraded to scanning radar.

2/28/08

CAM

When the WACR was powered up, the blower was not operating. T/S found the bearings were partially seized. Disconnected power from the blower and manually freed the fan to turn. Reconnected power and blower started running. Suspect bearings are near failure.

2/28/08

CAM

ProSensing engineer on site to reinstall WACR modulator and tube. Installed modulator and tube and reinstalled RF unit. Power tested system to be ok on 2/28. Returned to service on 2/29.

*Tube (Klystron) ProSensing S/N: 02 CPI S/N: E1011F5

*Modulator ProSensing S/N: 01

2/14/08

CAM

ProSensing engineer on site to remove modulator and tube from WACR to be returned to vendor for repair. Moved the transmitter from WACR shelter to the GIF trailer and removed the tube. Tube and modulator will be sent in for repair. Engineer will return to reinstall components after repairs are made.

12/18/07

CAM

The modulator continues to experience fault conditions. Instrument vendor requested the modulator power supply be removed from the rack, the bottom cover removed, and the power supply placed back in the rack upside down. Suspect coupling with the bottom cover may be inducing noise on the lines. System was already shut down at start of maintenance. Pulled the power supply from the rack and removed the bottom cover. Placed power supply back in rack upside down. Warmed the system up with SNMP.sh ON command and then restarted with the insectd command. Modulator high voltage power supply fault occurred shortly after startup. Shut the system down and returned modulator power supply to its normal configuration. Noticed pin 8 of J3 cable was pushed back into the connector at the power supply end and possibly not making good connection. Pulled the pin back out to its normal position and made sure it was secure. Ohmed out the cable and it tested ok. Restarted the system but the modulator faulted again. This time the message indicated the modulator faulted while warming up. Tried a couple more times with the same results. Also noted that the Fault 2 indicator on the modulator power supply does light briefly when the fault occurs. Notified vendor of results and system left powered off awaiting further instructions.

12/17/07

PAD

Last SDS collection was on 12/16/07 at 0210 GMT. Log shows Modulator body over current fault. Tried to restart instrument but Chiller over temp error is shown. Contacted ProSensing and was advised to start instrument but to not start taking measurements for 30 minutes to allow for instrument warm-up. Let instrument run until 1530 GMT and tried to restart with same results. Restarted instrument and let warm up until 1630 GMT. Found instrument not running at that time. Restarted and let warm up until 1714 GMT. Tried to start measurements and Modulator fault is immediately shown with instrument shut down. Notified Pro-Sensing. Troubleshooting at this time. RTS at 1730 GMT.

12/12/07

PAD

Last SDS collection was at 0410 GMT. Found computer has power but instrument is not operating. Started instrument at 1535 GMT and allowed instrument to warm up to operating temperature. Started radiating and data collection at 1611 GMT. RTS at 1615 GMT.

12/10/07

PAD

Last SDS collection was on 12/8/07 at 1410 GMT. Log shows chiller over temperature alarm at 1511 GMT on that date. Restarted instrument and raised set point on chiller controller from 15 C up to 20 C. Instrument quit after starting to take measurements. Called ProSensing and they remotely started instrument and was going to let water circulate through instrument for approximately 1 hour before taking measurements to allow instrument to warm up. RTS at 1601 GMT.

12/7/07

MK

Water needs to be drained from WACR chiller and replaced with 70/30 mixture of distilled water and propylene glycol. Shut down instrument and drained reservoir. Filled reservoir with mixture and restarted instrument. RTS at 1644 GMT.

12/7/07

PAD

Last SDS collection was at 0910 GMT. Instrument is running and radiating. Monitor shows READU error and graphs are not updating. Cleared error messages and closed out GUI box. Restarted GUI and graphs started updating. Shut down WACR to change chiller fluid. See other CM for details. RTS at 1620 GMT.

12/4/07

CAM

Vendor advised the system shut down again overnight, so lowering the temperature did not correct the modulator fault problem. Vendor reduced the PRF to 5kHz and will run it this way for a few days to see if faults still occur.

12/3/07

CAM

Need to replace the RF detector with the repaired original that was just received from the vendor. System was found already shut down due to a modulator fault 2 condition. Removed the RF unit side panel at 2020 GMT and R/R the *detector. Replaced the panel and restarted system. Lowered the chiller set point from 20C to 15C as suggested by vendor to try and resolve the modulator faults. RTS at 2100 GMT. *Old 353 (Hewlett Packard model WDH) *New 5E26 (Spacek Labs model DW-2N)

12/3/07

CAM

No data had been collected from the WACR since 1210 GMT on 12/1. Error message displayed: Warning The following error was encountered: READU; End of file encountered. Unit 5, File 127.0.0.1.2000. Please consult the supplier of the application. Closed error messages and restarted system at 1458 GMT. RTS at 1505 GMT

11/30/07

CAM

Continued T/S from 11/29 problem. The WACR computer will not boot up. Cycled the breaker on the UPS output to reset the whole system. The WACR computer still would not boot (computer power lights on, but nothing displayed on monitor). Tried power cycling the computer a couple more times with no change. Pulled the AC power cord from the back of the computer and reconnected. This time there was activity on the keyboard lights and a flashing cursor on the monitor, but nothing more. Pulled/reconnected the power cord again and got keyboard lights to flash and a brief glimpse of the boot up screen but then it froze again. Pulled the computer power cord, waited about 1 minute, and plugged back in. The computer booted up OK this time. Started the radar with the insectd command at 1831 GMT with external monitor still connected to the WACR computer. WACR computer then locked up again and displayed

Kernal Panic: Aiee, killing interrupt handler

In interrupt handler - not syncing.

Cycled power to the whole system once again. The computer booted up but indicated it did not mount the external data drive. Tried the firewire start command but could not mount the drive. Powered the entire system down, pulled the WACR computer out of the rack, and reseated all cards. Powered the system back on and the WACR computer booted up with no errors this time. Started the radar and RTS at 1935 GMT. System appears to be working ok for now.

11/28/07

11/29/07

CAM

SDS instrument status page indicated no data had been collected from the WACR since 1910 GMT. Checked instrument status and found system shut down with error message displayed: Warning The following error was encountered: READU; End of file encountered. Unit 5, File 127.0.0.1.2000. Please consult the supplier of the application. There were also several temperature, current, and duty cycle faults listed in one of the terminal windows. The last entry indicated the power strip was turned off at 1945 GMT. Closed all open applications and restarted the radar. Radar ran for a couple of minutes and then the radiate light went out. The displays stopped updating on the radar monitor computer. Tried to log into the WACR computer to stop the system but could not log in. Message indicated a path to host was not found. Unable to access WACR computer. Returned on 11/29 and connected a monitor and keyboard to WACR computer. Nothing displayed on monitor. Power cycled the WACR computer. Power lights are lit but still nothing displayed on monitor. Unable to access WACR computer remotely or locally. Left voice mail with Jim Mead at ProSensing. System remains down at this time.

11/28/07

PAD

Last SDS collection was at 1010 GMT. Log file shows on Nov 28 at 0405: body over current fault, over duty cycle fault, collector temp fault, detected fatal condition, sending children the term signal, turning off power strip. Restarted WACR. Notified instrument mentor and ProSensing via e-mail. RTS at 1543 GMT.

11/23/07

PAD

Error message displayed: Warning The following error was encountered: READU; End of file encountered. Unit 5, File 127.0.0.1.2000. Please consult the supplier of the application. Tried but was unable to restart WACR. External disk is found and radarop file is present. When I try to start the disk is shows: U must specify the file system type. Insect d command to start instrument replies: Cannot read config file, creating one. Left voice message with ProSensing and mentor. RTS at 1544 GMT.

11/20/07

CAM

The WACR system has returned from the vendor and needs to be installed. Installed WACR system at 1700 GMT on 11/20. Attempted to restart the system but the data drive could not be mounted. Contacted Jim Mead at ProSensing on 11/21. Jim restarted the system remotely at 1522 GMT and RTS status O.K. The WACR is currently running with a spare detector while the original is being repaired. The spare detector only reads about the actual power due to calibration differences. Site Ops will install the original detector when it returns from being repaired.

9/20/07

MK

The WACR system needs to be removed and sent off for repair. Removed WACR from service at 1500 GMT.

8/8/07

PAD

Last SDS collection was on 8/7/07 at 2110 GMT. Computer has power but instrument is not operating. Monitor computer shows normal errors displayed from shut down. After restarting WACR, instrument has power but is not radiating. Computer would not accept idl-vm command to set up monitor graphs. Called Pro-sensing and left voicemail. RTS at 1458 GMT.

8/7/07

PAD

Last SDS collection was on 8/6/07 at 1910 GMT. Computer has power but instrument is not operating. Chiller fluid level is visible. Monitor computer shows normal errors displayed from shut down. After restarting WACR, transmitter AVG power shows 484 W. Notified instrument mentor. RTS at 1410 GMT.

8/2/07

PAD

External data disk percentage used is the same as yesterday, 62%. Computer is not writing to files on external disk. Read through log when external disk was started on 7/31/07 after install of repaired instrument. Computer recognized disk but was not writing data to it. Found error in command to configure disk, data0 had been entered in command instead of data1. Stopped program and restarted external hard drive with correct command. Verified computer is now writing files to external disk. RTS at 1500 GMT.

7/31/07

MK

The WACR has returned from vendor repair and needs to be reinstalled. Started Installing WACR at 1700 GMT. Filled chiller with coolant, changed IP address with the help of Trent Doyle, powered on unit. After radar has warmed up it has been determined from the noisy data that there could be a problem with the tube. Jim Mead with ProSensing is currently working on the problem with tube manufacturer.

7/6/07

MK

Electronics rack an RF unit pulled and returned to vendor for modulator replacement.

6/25/07

MK

The WACR was not running at this time. Restarted WACR at 1645 GMT, system started with very low power. Contacted Jim Mead at Prosensing, he advised us to power off due to modulator failure. Unit will need to be sent back to vendor for modulator replacement.

6/4/07

PAD

External hard drive disk needs to be changed. At 1610 GMT, R/R external hard drive disk as per written instructions. RTS at 1625 GMT.

5/29/07

MK

The WACR is not running at this time due to a power outage over the weekend. Restarted at 1520 GMT. RTS at 1535 GMT.

5/3/07

PAD

External hard drive disk has reached 90% capacity and needs changed. R/R external hard drive disk at 1601 GMT. RTS at 1616 GMT.

4/4/07

PAD

R/R external hard drive disk at 1550 GMT. RTS at 1605 GMT.

3/27/07

MK

The replacement chiller fan has come in and needs to be installed. R/R chiller fan and restarted WACR. RTS at 1445 GMT.

3/20/07

PAD

Found instrument shut down. Computer and instrument has power. Chiller water temperature and RF Unit Heat Extraction graphs show spikes 24 C and 1000 watts respectively. Restarted instrument at 1432 GMT. Instrument shut down again within 15 minutes. Mark Klassen called Pro-Sensing and troubleshot problem with vendor. Found cooling fan in water chiller box is not working. New fan ordered. Instrument not in service at this time. RTS at 1502 GMT.

2/26/07

PAD

Last SDS collection was on 2/25/07 at 0710 GMT. Instrument has power but is not operating. Log file shows Chiller fault on Feb 25th at 01:59:20. Water is low but visible in reservoir. Filled water reservoir, checked for leaks, and restarted instrument. Instrument restarted correctly. RTS at 1549 GMT.

2/23/07

PAD

External data disk is 87% capacity and needs to be replaced. Shut down instrument and R/R external data disk at 1701 GMT. Restarted program and restored monitor graphs. RTS at 1716 GMT.

11/1/06

PAD

Mentor has requested external hard drive disk changed when it reaches 90% capacity. Ron Reed stopped program at 2015 GMT and changed external disk as instructed. Removed disk SN 162203459 and installed disk SN 151802302. Restarted program. RTS at 2030 GMT.

10/2/06

MK

The system need to be installed after the vendor made upgrades. Installed the transmitter and equipment rack and powered on at 2045 GMT.

9/13/06

DKB

The system need to return to the vendor for upgrades. Powered off the instrument and removed the transmitter and equipment rack for shipment to Pro Sensing.

6/2/06

MK

The body current meter was not measuring any current. Contacted Jim Mead with Pro Sensing, he remotely monitored the radar and verified that the radar is still functioning properly. He suggested that when the RF unit goes in for upgrade that the power supply unit be pulled and sent in for a meter replacement.

4/13/06

DKB

Stopped data sampling, reduced RF power and performed calibration test. Returned RF power to normal and restarted data sampling at 2025 GMT.

4/13/06

DKB

Replaced the radar target with a larger one and performed system range calibration checks with the mentor. Stopped data sampling, reduced RF power and performed calibration test. Returned RF power to normal and restarted data sampling at 1825 GMT.

3/31/06

PAD

External hard drive disk is at 86% and needs to be changed. At 1542 GMT, called James Mead with Prosensing to receive instructions on changing hard drive. R/R hard drive as per instructions. James was restarting instrument remotely at 1622 GMT.

3/24/06

CAM

Received the transmitter and controller and need to install. Installed the transmitter and leveled. Installed controller. Contacted Jim Mead at ProSensing as requested before powering on. Powered system on as per instructions and RTS status O.K. at 1925 GMT.