What are Quality Assurance and Quality Control (QA/QC)?

QA/QC basically refers to all those things good investigators do to make sure their measurements are right on (accurate; the absolute true value), reproducible (precise; consistent), and have a good estimate of their uncertainty. In the regulatory arena, this aspect of data collection is as crucial to the final outcome of a confrontation as the numbers themselves. It specifically involves following established rules in the field and lab to assure everyone that the sample is representative of the site, free from outside contamination by the sample collector (no dirty hands touching the water) and that it has been analyzed following standard QA/QC methods. This typically involves comparing the sample to a set of known samples (calibration standards) for estimating accuracy and by replicating the measurement to estimate its precision. The U.S. Environmental Protection Agency and the U.S. Geological Survey are federal agencies that have lots to add should you wish to learn more of the technical aspects of a Quality Assurance Program (QAP) and creating a Quality Assurance Project Plan (QAPP).

Proper Quality Assurance and Quality Control (QA/QC) protocols are essential to the LakeSuperiorStreams project and therefore we have gone to great lengths to try to assure the accuracy of our data. Quality Assurance is a particularly important component of any program that posts real-time or near-real time data on a Website that is freely available to the public. Data for this project will be collected primarily by LakeSuperiorStreams staff from the Natural Resources Research Institute (NRRI) at the University of Minnesota-Duluth (UMD) with additional help from the City of Duluth Stormwater Utility, the Minnesota Pollution Control Agency (MPCA), and the Western Lake Superior Sanitary District (WLSSD).

The methods used for data collection, storage and retrieval as well as quality control and data delivery will closely follow those described in the EPA-EMPACT Program's technology transfer manual based on the Lake Access - Minneapolis Project (EPA 2000). This project was developed by our NRRI-UMD group working in collaboration with limnology staff at the Hennepin Parks - Three Rivers Park District in the Minneapolis, MN Metropolitan area. Other aspects of the "flow" of information from real-time "data hoses" are described in Host et al (1999, 2000).

DATA TYPES

There are basically four sets of environmental data that are collected for LakeSuperiorStreams:

1. Water quality parameters analyzed in the lab from "grab" samples of water such as nutrients (N- and P-series of nutrients), pH and alkalinity, turbidity, total suspended solids (TSS), total dissolved solids (TDS), color, chloride, fecal coliform bacteria, total volatile solids (TVS) and biochemical oxygen demand (TVS and BOD5 are both measures of organic matter);
2. Water quality measurements made in the field at the time of water sample collection such as sensor measurements of temperature, water velocity (to estimate flow), dissolved oxygen (DO), specific electrical conductivity (EC25) and transparency tube measurements of water transparency or clarity, etc.; and
3. Remotely sensed water quality measurements collected, logged and transmitted by electronic sensors deployed in the streams that we call SMU's for Stream Monitoring Units. These instruments are programmed to record temperature, stream elevation (also called stage height), EC25 (a measure of total salt concentration), and turbidity (used as a measure of suspended sediment). In addition, the SMU at the Kingsbury Creek site is connected to an automated sampling device (ISCO 6712) that upon preset increases in stage height, signifying a storm event, initiates sample collection of up to 24 individual water samples into one liter plastic bottles. We have initially programmed the unit to sample at 2 hour intervals during storm events.

   The stream sensor data is typically collected every 15 minutes, stored throughout the day and then transmitted via cellular phone/modem to our base computer/website each morning. We can collect the data more frequently, and call up the unit to see what the latest data looks like, but this adds to our phone bill. A final intensive data set is collected at the Duluth-Superior Harbor/ St. Louis River outlet to Lake Superior where a sensor array measures and logs temperature, EC25 and turbidity at 15 minute intervals. This unit was installed on the channel wall above the USGS Superior Bay Duluth Ship Channel (Station 464646092052900) where an acoustical velocity meter (AVM) system with a two-path transducer installation is used to measure river discharge into Lake Superior. We have previously established a data-sharing partnership with USGS (Madison, WI, P. Hughes) to link our on-line water quality data with their discharge data.

4. Volunteer monitoring data. During this grant period, LakeSuperiorStreams staff will work with the MPCA to coordinate existing volunteer monitoring with the goal of achieving more consistent methodology and developing a repository and database for stream-related information in the City of Duluth. Programs include the state-wide Citizens Stream Monitoring Program (MPCA sponsored), St. Louis River Watch and a variety of less formal efforts by local schools and environmental learning centers. LakeSuperiorStreams initiated school-based stream monitoring programs at sites on Chester, Tischer and Kingsbury Creeks in October 2002 as part of the National Monitoring Day, nation-wide celebration of the 30th anniversary of the Clean Water Act on October 15. It was subsequently decided to include these schools in the St. Louis River Watch program which already includes over 30 area schools (further information).

   Parameters will vary depending on the level of effort available, but at a minimum will include temperature, transparency tube depth, turbidity and EC25 during seasonal baseflow conditions, snowmelt runoff and rainstorm runoff. Training will be provided by River Watch and LakeSuperiorStreams staff, and turbidity and EC25 measurements will be performed by NRRI on discrete samples collected by the schools. Biological monitoring of macroinvertebrate communities is fundamental to River Watch, and LakeSuperiorStreams will work to encourage standardized sampling protocols that conform to peer-reviewed research standards and existing state and federal guidelines. The goals of this effort are to initiate a comprehensive database, standardize monitoring methods, and educate students and citizens.

Conventional data quality assurance procedures follow guidelines set by the U.S. EPA (1989a,b), and the so-called "Standard Methods" from APHA (1998). Water chemistry and manual field profiles are collected by trained staff aquatic ecologists and technicians at NRRI-UMD (supervised by Co-Principal Investigator / Limnologist Rich Axler) and Minnesota Sea Grant-UMD (supervised by Co-Principal Investigator / Aquatic Ecologist Carl Richards). City of Duluth Stormwater Utility technicians have worked closely with UMD staff to ensure that consistent sampling and SMU maintenance procedures have been used. All water chemistry analyses are performed by either the NRRI Central Analytical Laboratory or the WLSSD laboratory which are certified annually by the Minnesota Department of Health for Federal Safe Drinking Water Act and Clean Water Act parameters. The certification procedure involves blind analyses of certified performance standards and an in-depth site inspection and interview approximately every other year. The NRRI lab has also been certified over the past decade by the Minnesota Pollution Control Agency and the Minnesota Department of Natural Resources for low-level water quality analyses in pristine, acid-sensitive lake monitoring programs and for sediment contaminant analyses in the St. Louis River and Upper Mississippi Rivers. Further details regarding the water chemistry analyses are provided below.

SENSOR RESOLUTION & REPORTING LIMITS

Ideally, if all of the sensors behaved according to the sensor manufacturer's specifications (Table 1) we could simply post the data on the LakeSuperiorStreams web site and assume it is accurate to these levels. However, except for temperature, all of the sensors require routine maintenance and calibration. When visiting sites we re-calibrate the EC and turbidity sensors using individual standard solutions with known values for EC25 and turbidity. Experience has taught us that the sensors remain stable during the course of a sampling day. The depth sensor is also calibrated at this time to read zero when the sensor is set at the water surface.

When deployed for continuous operation the sensors are colonized gradually by a biofilm of algae and less noticeably, by bacteria and fungi as well. As this material builds up, the biofilm interferes with the sensor's ability to accurately sample the surrounding water. One can easily picture fine filaments of algae, like you see on the rocks in summer, wafting intermittently across the electrodes of the EC sensor or in the light path of the turbidimeter. This would produce erratic values with wide swings even during periods of low and relatively constant flow when the stream is running clear. Turbidity probes are most susceptible to these changes, followed by dissolved oxygen (not measured automatically at this time) and EC. Fine particulate sediment will also be trapped and gradually contribute to erroneous readings if not cleaned on a regular basis. Turbidity sensors are "notorious" for their maintenance problems - the YSI 6136 sensor we are using is called a "self-wiping" sensor that mechanically wipes the optical window used for the measurement just before a reading is taken. Although this feature certainly reduces errors due to fouling, the wiper occasionally "sticks", creating apparent "spikes" in the data set (see below). The Hydrolab self wiping turbidity sensors have not experienced these wiper sticking issues as of yet.

Other sources of variability include the natural variations that occur in a stream due to eddies and larger debris suddenly breaking free, events less likely to be seen in the calmer water in deeper stream pools or in lakes. Also, the sensor is actually "sampling" only a tiny (millimeter scale) patch of water for less than a second every 15 minutes. Table 2 compares the theoretical sensor resolution with our best estimate of the true accuracy of the sensor readings, incorporating all sources of known error.

Stream Monitoring Unit (SMU) PLACEMENT, FLOW CALIBRATION , SAMPLE COLLECTION

We attempted to place the water quality probes at representative measurement points in the stream cross-section with allowances made for protecting the units from debris and sediment where necessary. Probes are inspected, calibrated, and maintained following the manufacturers recommendations in addition to following USGS (2000a, 2000b) procedures. The exact location of each unit within a watershed was chosen based upon considerations of security, stability regarding anticipated road and bridge construction activities, and on upstream land use characteristics. Because of this, summary interpretations as to water quality differences between streams must be done with caution since such a direct comparison was not intended.

Stream discharge is determined as in Anderson et al (2000) and USGS (2000b) with rating curves developed and subsequently used to calibrate flow. Stream depth in Tischer, Chester and Kingsbury is measured remotely using a pressure transducer and discharge is determined using rating curves based upon a set of cross-sectional and discrete depth in-stream velocity measurements made with a Marsh-McBirney stream velocity meter over a range of discharge conditions. Stream depth at Amity and Poplar is measured by an MPCA owned/operated/maintained ultrasonic distance sensor and discharge calculated from a USGS derived flow rating curve.

Discrete water samples are collected at Kingsbury Creek at various points along the hydrograph using an ISCO 6712 slaved to the YSI 6820 stream elevation sensor. Periodic intensive manual collections throughout high water periods as well as during baseflow periods are conducted for Chester, Tischer, Amity and Poplar Creeks.

SENSOR CALIBRATION

NRRI staff follow the Instrument Manuals for calibration and maintenance procedures. Our staff also have extensive (i.e. decades collectively) experience with these calibration procedures and with their importance in interpreting field data and distinguishing systematic errors associated with deteriorating, or bio-fouled probes. Our Lake Access, EMPACT project is a companion to an earlier NSF-funded Advanced Technology Education project entitled Water on the Web (WOW: http://WaterOnTheWeb.org), now in its fifth year, that involved establishment of Apprise Technologies, Inc. robotic water quality sensing RUSS units on five Minnesota lakes. Since 1998 we have gained extensive experience in dealing with the problems associated with continuous sensor deployment and the resultant protocols are included also in the Lake Access and WOW websites.

The following protocols have been set up to minimize these biofouling and instrument drift effects to quality assure the LakeSuperiorStreams data:

• Clean and re-calibrate sensors frequently (about every 2 weeks) and perform manual measurements with an independent instrument at the same time (temperature and EC25) or soon after in the lab (turbidity). The depth sensor is also calibrated at this time by re-setting it to equal zero depth when placed just above the water surface.
• Compare independent manual measurements with near-simultaneous SMU data prior to cleaning (re-calibration). This provides assurance that the previous period of data is accurate. We calculate test statistics for each parameter as:

  RPD (relative % difference) = |Manual-SMU| x 100 , and
  [SMU]

  DELTA = |Manual – SMU| for each parameter. They PASS according to rules in Table 3.

  Table 3. Quality Assurance Criteria for SMU Sensors
  SENSOR	RPD	DELTA
  temperature	< 5%	< 0.2 °C
  EC25	< 10%	< 5 µS/cm
  turbidity	< 10%	< 5 NTUs
  depth

If the data "passes," it is considered acceptable for the previous period. If not, we examine it in the context of our understanding of the instrumentation, the stream's prior automated and "manual" water quality data, its watershed and the recent weather pattern and then either delete it from the database or allow it to be posted. We have to be careful not to delete anomalous data that may simply reveal real dynamic changes. The sheer volume of data has been taxing and we lack the resources to always be as current as we would like. In the interim, data are posted as provisional. Dates of calibrations and manual data are both posted in the DATA section of the website and are available within easily accessible Excel files. The stream data visualization tool (DVT) is also helpful in rapidly displaying the data in a variety of formats to help identify anomalous data. Calibration "date flags" will be added to the control panel of the DVT to allow the user to more easily keep track of calibration dates as the data stream is being viewed.

Although not yet implemented (as of December 2002), we are also exploring a data quality classification rating system such as used by the US Geological Survey for certain continuous water quality records. The table below is taken from USGS (Wagner, R.J., Boulger, R.W., Jr., Oblinger, C.J., and Smith, B.A., 2006, Guidelines and standard procedures for continuous water-quality monitors—Station operation, record computation, and data reporting: U.S. Geological Survey Techniques and Methods 1–D3, 51 p. + 8 attachments; accessed April 10, 2006, at http://pubs.water.usgs.gov/tm1d3 - Table 18).

This USGS document also discusses the use of such data when there are extensive periods without data as well as other considerations that are beyond the scope of our present objectives.

OTHER CONTINUOUS WATER MONITORING DATA

The MPCA, in collaboration with the South S. Louis County Soil and Water Conservation District (SSLCSWCD) has also operated several continuous stage height monitors on Miller Creek, and these data will also be posted. A limited number of water quality samples have also been collected at the St. Louis River/Lake Superior entry site.

DATA TRANSMISSION AND INITIAL QA SCREENING

Each SMU is called daily and data that has been collected since the last call is downloaded. These new data files are stored on the base station computer as comma-delimited ASCII text files. The Kingsbury station is polled twice daily, at 6 AM and 2 PM. The Chester and Tischer sites are scheduled to be polled at 6AM each day because of cell phone costs.

The Conversion Process
A program (the data importer) is now launched. It reads any data files that have been created or changed since the last time it was run, and converts the data to the format used by the report generating and data visualization programs. The data importer examines the beginning of these files and checks to make sure that the contents (e.g. site name, parameter names and units) correspond to what is expected. If not, an error message is generated and no further action is taken with this file. This will catch errors that could occur if, for example, a new parameter is being read by the SMU, but the property files used by the data importer for that site haven't been updated to handle the change.

Now, each data line is read and converted to the "DataTable" format used for storage. The importing program can reject specific data if it is outside of a pre-defined range for the given parameter. Data are automatically rejected if outside the following ranges:

If it encounters a value outside of the range it will generate an appropriate error message in the log file, and disregard the value. This helps eliminate invalid readings that could be caused by sensor drift or SMU hardware problems. After all of the new data files have been imported the data importer triggers the appropriate programs that will create and send updated reports and data-visualization data files to the website.

FINAL DATA REVIEW & POSTING

At present (December 2002), funding limitations have precluded adherence to a rigorous schedule for removing the provisional label from LakeSuperiorStreams automated data. In part this is due to the need to review ancillary water chemistry data before making final decisions when the data is questionable. All manually sampled water quality data posted on the LakeSuperiorStreams however, have passed QA/QC prior to being posted, although this typically takes from 30-60 days after collection.

Despite regular maintenance and calibration schedules, occasional SMU data anomalies still occur. To date, they have generally been associated with the turbidity sensor although there was great difficulty during summer 2002 in getting the stage height sensors to function properly. Additional difficulties were caused by the Minnesota DNR modifying the Chester Creek stream channel in the vicinity of our SMU, causing excessive turbidity values and requiring a new empirical calibration of the stage height-flow relationship. There have also been start up problems associated with precipitation monitoring and the modem link to Kingsbury Creek. All of these problems were resolved by about September 1, 2002.

The most troublesome anomalies are those that occur within the calibration window of time, are not flagged by our automated screening tools and are not unreasonable values in terms of the range of values previously measured or expected for the hydrologic conditions at the time. These errors will not be trivial to identify and will require careful examination in a complete fluvial/limnological (stream/watershed/climate) context by professionals. The process is adequately described as Best Professional Judgement (BPJ). In some cases data will need to be adjusted by calculating correction factors when there is accurate calibration data spanning the period in question and when the results estimated by interpolation are consistent with the rest of the data set. In other cases, it will be necessary to simply reject the data - omitting it from the website. A log of data deletions will be maintained on the website, and an e-mail announcement sent to all teachers and researchers known to be using the site for educational or research purposes related to curricula associated with LakeSuperiorStreams and Water-on-the-Web (http://WaterOnTheWeb.org).

Data collected under ice: When streams are ice covered the depth readings and flow calculations are subject to errors:

• The stream can be running under the ice without an air gap, essentially pressurizing the stream -- increasing the apparent depth and velocity.
• The streams can be running on top of the ice (probably unaccounted for flow).
• Anchor Ice may change the channel cross section, making our rating curves unreliable.
• Ice dams below the site can change flows.
• Bank ice can constrict the channel.

All of these things can happen at the same place over the course of time as well.

We have chosen to flag this data as 'collected under ice cover'. It will still show relative changes, especially over the short-term, but it should be noted that the reported depths and flows are not necessarily accurate.

WATER SAMPLE ANALYSES

Data from the monitoring programs will follow established procedures of the NRRI Central Analytical Laboratory (Ameel et al. 1998) and WLSSD (2000a,b). Both laboratories are annually certified for Clean Water Act and Safe Drinking Water Act water quality parameters by the Minnesota Department of Health.

Sample Collection: Samples for micronutrients ([nitrate+nitrite]-nitrogen, ammonium-nitrogen, total-nitrogen [TN], orthophosphate-phosphorus [OP], total phosphorus [TP], alkalinity, chloride, BOD5, total suspended solids [TSS], total volatile solids [TVS], and total dissolved solids [TDS] and fecal coliform bacteria are collected exclusively by trained personnel (City of Duluth, WLSSD, MPCA, NRRI Cental Analytical Laboratory personnel), according to individual analysis requirements using an ISCO 6712 automated sampler at the Kingsbury site. Manual samples are collected at various times of the year to characterize Chester and Tischer Creeks at their SMU sites. Several intensive sampling surveys will be conducted to characterize high flow storm events and the snowmelt runoff period in the Spring. Grab samples for fecal coliform bacteria will be collected using sterile technique (APHA 1998). All samples are transported to the laboratory on ice in dark coolers on the day of collection, within a few hours after collection. A field data sheet accompanies each sample. Sample sites for the collection of duplicate samples and preparation of field blanks are chosen at random. Manual samples at Amity and Poplar river are being collected by the MPCA.

Sample Preservation: All samples are initially preserved by refrigeration and when filtration is necessary it is performed as soon as possible after return to the lab. If delay in analysis is anticipated, additional preservation will be incorporated, as provided in individual methods and according to approved guidelines (e.g. acidification or freezing). The container label is marked when preservative is added.

Sample Identification and Tracking: Sample containers are labeled in the field with date and time of collection, location of sample, and person collecting the sample. Field data sheets include all of the above information, and also conditions specific to the project and lists the individual parameters for which analysis is requested. For stream monitoring, and other sampling events for which field analysis are performed (such as conductivity, dissolved oxygen, pH, temperature), an additional sheet is used which lists site, stream conditions and other pertinent information. Water samples delivered to the laboratory are assigned a laboratory identification code and entered into a computerized database (Microsoft Access) along with location, collection date and required analytes. The code is used to keep a running tally of all samples received, and also to track analysis as they are completed.

Instrumentation: Laboratory instruments are calibrated, operated, and maintained properly according to manufacturer's recommendations and this is a requirement for State Laboratory Certification. Instruments are calibrated each time they are used, if applicable. A log is kept of each instrument where maintenance and general comments on instrument performance are documented. For spectroscopy procedures, the calibration is performed with reagent blanks and standards of a matrix closely matching the samples. A typical standard curve consists of at least five concentrations. The linearity of each standard curve is monitored and must have a regression coefficient (r2) greater than 0.99. Additional quality assurance procedures that are a part of every micro- and macronutrient run include blanks, QCCS (quality control check standards), and internal spikes (recoveries of 80-120% or 85-115% are typically required depending on analyte).

Data Storage and Archiving: After analysis, raw data from labsheets are entered into spreadsheets to calculate concentrations and compute the relevant QC statistics. The analyst checks the quality control data against laboratory limits, and if the data meets the requirements, the data is saved and printed in hard copy. Both the original data sheet as well as the spreadsheet printout are saved. On the hard copy, the analyst checks the data, shows the quality control calculations, and initials the completed sheet. This data is reviewed by the Laboratory QA officer who also initials the sheet. At this point the data is considered valid and reportable. Data from spreadsheets is imported electronically into the water quality database on the LakeSuperiorStreams-NRRI file server. The server is backed up daily to prevent data loss.

REFERENCES

Ameel, J., E. Ruzycki and R.P. Axler. 1998 (reviewed annually). Analytical chemistry and quality assurance procedures for natural water samples. 6th edition. Central Analytical Laboratory, NRRI Tech. Rep. NRRI/TR98/03.

Anderson, J., T. Estabrooks, and J. McDonnel. 2000. Duluth Metropolitan Area streams snowmelt runoff study. Minnesota Pollution Control Agency, St. Paul, MN. March 2000.

APHA.1998. Standard methods for the examination of water and wastewater. Amer. Publ. Health Assoc.

EPA. 2000. Delivering timely water quality information to your community: The Lake Access-Minneapolis project. EPA/625/R-00/012, September 2000, U. S. Environmental Protection Agency, Office of Research and Development, Cincinnati, OH, 45268, USA.

EPA. 1998. Guidance for Quality Assurance Project Plans EPA QA/G-5. EPA/600/R-98/018, Feb 1998 (http://www.epa.gov/quality1/qs-docs/g5-final.pdf ). U.S. EPA, Washington, D.C. 20460

EPA. 1996. The Volunteer Monitor's Guide to: Quality Assurance Project Plans. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA (http://www.epa.gov/region09/qa/pdfs/vol_qapp.pdf)

EPA 1989a. Preparing perfect project plans. US EPA Risk Reduction Engineering Laboratory, Cincinnati, OH, EPA/600/9-89/087.

EPA.1989b. Handbook of methods for acid deposition studies-Field operations for surface water chemistry. Environmental Protection Agency, Washington, D.C.

Host, G., N. Will, R.Axler, C. Owen and B. Munson. 2000. Interactive technologies for collecting and visualizing water quality data, URISA Journal (http://wow.nrri.umn.edu/urisa)

Host, G.E. , B. H. Munson, R. P. Axler, C. A. Hagley, G. Merrick and C. J. Owen. 1999. Water on the Web: Students monitoring Minnesota rivers and lakes over the Internet. AWRA Spec. Ed. (Dec., 1999). (refereed: http://www.awra.org/proceedings/www99/w74/index.htm

USGS. 2000a. Guidelines and standard procedures for continuous water-quality monitors: Site selection, field operation, calibration, record computation, and reporting. R.J. Wagner, H.C. Mattraw, G.F. Fritz and B.A. Smith. U.S. Geological Survey Techniques of Water-Resources Investigations Report 00-4252 (http://water.usgs.gov/pubs/wri/wri004252/). U.S. Geological Survey, Reston, Virginia, USA.

USGS. 2000b. National field manual for the collection of water-quality data. U.S. Geological Survey Techniques of Water-Resources Investigations, book 9, chaps. A1-A9, 2 v., variously paged.

WLSSD. 2000a. Laboratory Procedures Manual. Revision 2, Oct 1998. Western Lake Superior Sanitary District, Duluth, MN. 55807.

WLSSD. 2000b. Laboratory Quality Assurance Manual. (Revision 2, May 1998 with annual minor revisions). Western Lake Superior Sanitary District., Duluth, MN 55807.

updated: October 4, 2018