UNIVERSITY OF CALIFORNIA, SAN DIEGO
.
..........www.
.
www.
wewww..wwwwwwwwwwwwwwwwwwwwwwwwwwwww
www.iwawole
> BWAWAU
www.....
....
..
UC SAN DIEGO LIBRARY
.......
..
www
.
.
3 1822 04429 7505
Feb 1991
OPTICAL SYSTEMS GROUP
TECHNICAL NOTE NO. 226
Offsite
(Annex-Joi
rnals)
QC
974.5
. T43
no. 226
WSI DATA BASE SUMMARY
(STATUS AS OF 31 DEC 1990)
R. W. Johnson
J. E. Shields
T. L. Koehler

....
....
..
.
.........
UNIVERSITY
OF
CALIFORNIA
SAN DIEGO
.
..
...
.
.
........
..
RSITY
www
SMIN
CA
Contract Monitor, Dr. J. W. Snow
Atmospheric Sciences Division
:
#
III
*WWUNIT
Kahiturak
#
ORNA
1868
Prepared for
The Geophysics Directorate of the Phillips Laboratory
Air Force Systems Command, United States Air Force
Hanscom AFB, Massachusetts 01731
under contract No. F19628-88-K-0005
SCRIPPS
INSTITUTION
OF
OCEANOGRAPHY
MARINE PHYSICAL LAB San Diego, CA 92152-6400

UNIVERSITY OF CALIFORNIA, SAN DIEGO
ww
w
www............
w
.
wwwwwww
.....
.
w
.
...
.w.
..
www.
w
.
w.www.ww
.....
arctica
wwwwwwwwwwwwwwww..
.....
....
.
w wwwww...
.
w w
puwede
.
w
.
3 1822 04429 7505
VO
Technical Note No. 226
WSI Data Base Summary
(Status as of 31 Dec 90)
Summary
Responding to a well recognized need by many in both the modeling and
operational communities for an improved capability in the collection and
assessment of wholc sky cloud characteristics, a new generation of video
based imaging systems has been developed and fielded by the Marine
Physical Laboratory. Onc of thcsc systems, the Whole Sky Imager, has been
deployed at several widely separated portions of the United States, and has
gathered scveral million images appropriate for determining cloud cover at
very high spatial and temporal resolution. Cloud cover estimates derived
from a 7-month sample of these cloud images shows very good agreement
with observed sky cover amounts. The capabilities of the Whole Sky
Imnager are discussed, followed by an overview of the current status and
quality of the WSI data basc.
WSI DATA BASE SUMMARY
STATUS AS OF 31 DEC 90
Table of Contents
1.0
Introduction ......
2.0
Automa
Automated Systems for Cloud Assessments ....
2.1 Whole Sky Imager........
2.2 Real Time Cloud System .....
2.3 Portable WSI........
2.4 Supporting Instrumentation ...
3.0
WSI Cloud Data Archive ........
3.1 Background ..
3.2 Current Extent of the Data Base..
Standard WSI Data Processing Procedures .....
4.1 Data Processing Procedural Sequence...
4.2 Data Processing Run Time Estimates ..
4.3 Data Processing Status .....
ovu vua au AA www wWN--
5.0
Alternative Products......
5.1 Upgrade CLDDEC ........
5.2 Create Calibrated Radiance Library .....
6.0
WSI Data Evaluation ....................
6.1 Comparison with Observer ....
6.2 Cloud Cover Temporal Dynamics Illustration ....................
7.0
Conclusion......
8.0
Acknowledgements
9.0
References & Bibliography ......................................
WSI DATA BASE SUMMARY
STATUS AS OF 31 DEC 90
SSR
List of Illustrations and Tables
Fig. No.
Title
Page No.
:
:
2-1
2-2
2-3
2-4
2-5
2-6
2-7
Whole Sky Imager Camera Assembly .
Sample WSI Image, acquired at 650nm
Sample Cloud/No Cloud Decision Image, 10 minute ..........
Sample One-Minute Cloud/No Cloud Decision Image ..............
Sample Image from Real Time Cloud System ....
Portable Whole Sky Imager ..........
..........
Image Acquisition & Analysis System ......
en w NNN-
3-1
WSI Data Base
bent
4-1
4-2
WSI Basic Image Processing Flow Chart........
Raw Imagery to Decision Imagery Conversions..
WSI Data Status & Processing Summary
sebenar
S
.
4-3
5-1
v
Raw Imagery to Calibrated Radiance Data .......
6-1
6-2
6-3
Distribution of Total Cloud Cover Deterininations .................
Total Cloud Cover Determination Differences ..
Total Cloud Cover Time Series ...............
WSI Cloud Cover Image at 1420 ............
WSI Cloud Cover Image at 1450 ............ ...........
o oo oo oo oo
6-4
6-5
Table No.
3.1
WSI Data Tape Status as of 31 Jan 91 .....
WSI
A
4.1
4.2
WSI Standard Data Processing Configurations .....
Raw to Decision, Run Time Summaries .............
Processed Data Tape Summary ......
aaa
4.3
..........

1.0 INTRODUCTION
A family of imaging systems for use in automated
cloud assessment and sector visibility determination has
been developed by the Marine Physical Laboratory of
Scripps Institution of Oceanography (Johnson, 1989).
This paper will give a brief overview of two of these
systems, the Whole Sky Imager (WSI) for long term
cloud studies and the Real Time Cloud system for on-line
operational support. Each of these systemsis an automated
unit operating under micro-computer control, gathering
digital imagery suitable for automated processing and
analysis.
The WSI systems have been operating at several
locations during the last few years, acquiring images
once a minute, 12 hours a day. These data are archived
and subsequently converted to cloud/no cloud decision
images. The resulting cloud database can be utilized for
a variety of applications including model evaluation and
development (Hering, 1989). We will review the acqui-
sition and processing of the WSI cloud database, and
discuss some of the emerging statistical relations from
the WSI cloud determinations.
Fig. 2-1. Whole Sky Imager Camera Assembly
2.0 AUTOMATED SYSTEMS FOR CLOUD
ASSESSMENT
2.1 Whole Sky Imager
The Whole Sky Imager (WSI) is a ground-based
electronic imaging system, which monitors the upper
hemisphere. It is a passive, i.e. non-emissive system,
which acquires calibrated multi-spectral images of the
sky dome. The system, shown in Fig. 2-1, views the sky
through a series of spectral and neutral density filters,
using a fisheye lens to acquire most of the sky dome. A
fixed gain CID (charge injection device) solid state
camera is utilized, and a full set of radiometric and
geometric calibrations are acquired prior to fielding the
system. Data are acquired in 512 x 480 format, which
yields 1/3 degree spatial resolution. This corresponds to
a 17 meter footprint, for a cloud layer at 3 km height.
In the field, cloud images are acquired under control
of an IBM AT-class microcomputer. This is a stand-
alone unit, requiring essentially no user intervention;
control of all peripheral functions is fully automatic.
Four digital images are acquired every minute, and
archived on 8 mm tape, for post-processing. Approxi-
mately 1.2 gigabytes are acquired and archived per week
at each site.
Fig. 2-2. Sample WSI Image, acquired at 650 nm
are near the horizon. The black square is the sun
occultor, which shades the lens in order to minimize
stray light. A tower may also be seen in the field of view
near the occultor. In the processing, a determination is
made at each point in the image, yielding a cloud
decision image at full spatial resolution. The cloud
decision image is illustrated in Fig. 2-3. In this illustration
the areas identified as sky are blue, the occultor is black
indicating a "no data" region, and the pixels identified as
thin or opaque cloud are yellow and white respectively.
A sample radiance image is shown in Fig. 2-2. In this
image, the center is at the zenith overhead, and the edges

SKY COU
all
field spatial and temporal dynamics. The WSI acquires
calibrated multi-spectral sky radiances every minute.
From these, automatic cloud determinations are pro-
duced with 1/3 degree spatial resolution. The system was
originally deployed at 7 sites throughout the country, and
as of 1 Jan 91, had acquired an average of 26 months of
data per site. The data will be further discussed in
Section 3.
2.2 Real Time Cloud System
An outgrowth of the WSI is the Real Time Cloud
system, which provides automatic cloud assessments in
real time in the field, for on-line operational support.
This system acquires sky radiances, much like the WSI.
The user may input a track of interest, such as the track
of a satellite or drone. The cloud determinations are then
made from the measured radiances and presented to the
user in color-coded image format, i.e. blue for sky, white
for opaque cloud, and yellow for thin cloud as shown in
Fig. 2-5. In the user's display, the track is color coded by
Fig. 2-3. Sample Cloud/No cloud Decision Image.
This shows the full resolution results taken
at 10 minute intervals.
18籍路超
​15000
Full resolution images such as shown in Fig. 2-2 are
saved every ten minutes. At one minute intervals, the
subset shown in Fig. 2-4 is saved. (Fig 2-4 is the one
minute cloud decision image.) This image consists of a
subset of rows and columns, each saved at full resolution
(i.e. we save 33 rows, and 33 columns, for 33 x 512 + 33
x 480 resolution). Thus for cloud free arc length statis-
tics, we have full resolution, but for cloud areal statistics
such as cloud cover, we have reduced resolution in the
one minute data. Our early studies indicated an uncer-
tainty of approximately a 1% in the cloud covercomputed
from the one minute data as compared with the ten minute
full resolution data.
COLUMPIRMO
SHY COU 63
AP80 20 30
70
p
rogramas
wwwddddd
www.domeny
d
ecedere de propongo
december
redde prendre
www.
d
om
t
dodh
e
chem.c
program
om
donderwerpen
www.
downl
o
ad
မျိုးရေးခွဲ willwww
w wwwwwwwwwသန်းwwwwwwwwwwwwwwxမိုး/www/www) /wwwမဲ့၊ န်း ၊
wwwdelme
wedderendum
pengendaliano
gondolatry
w id
endido
propriedade de
Twinlay) wwwwwwwwwwwwwww
ww w . nyilawhn a wwwwwww1) |
100kmadieniowanie moghting
wwwwwwwwwwwwww
modern
domendado
p
d
e
marajanje
gewoon
re
'S MY COUER ESTIMATE
SMY COUE
OpH7
Fig. 2-5. Sample image from Real Time Cloud System
Note user-input track,
altitude, and superimposed on the image. Percent cloud
cover, both total and opaque, are presented to the user for
both the sky dome and specifically along the track of
interest. If desired, the user may view previously stored
images, which are video looped to display temporal
changes in the cloud cover. The current system is
connected by optical fiber to a WSI unit, and has its own
micro-computer for independent image acquisition and
display. The system is currently deployed in New Mexico,
for support of the HELSTF laser site, however it has
application to a wide variety of operational and test
scenarios.
Fig. 2-4. Sample One-Minute Cloud/No cloud
Decision Image. This shows the subset saved
1 minute intervals.
In summary, the Whole Sky Imager is a system for
cloud assessment, specifically for the archival of cloud

2.3 Portable WSI
The portable WSI unit is shown in Fig. 2-6. This unit,
designed for short term deployments and/or special
applications, is similar to the WSI, but has a more
transportable support and environmental control hous-
ing. The portable system was initially installed at the
University of Wisconsin, for a cooperative study with
Eloranta and Grund, using their HSRL lidar system.
The intent was to relate the WSI cloud determinations to
the cloud optical depth determined by the lidar system.
point array processor to accelerate the system through-
out, and multiple EXABYTE drives are attached to each
control computer.
3.0 WSI CLOUD DATA ARCHIVE
The Whole Sky Imager has been operating routinely
at several sites in the continental US for over two years.
This section discusses the background of this data archive,
and then reviews the status of the data processing and
cloud algorithms.
3.1 Background
Fig. 2-6. Portable Whole Sky Imager
2.4 Supporting Instrumentation
The WSI cloud data base is stored in digital format on
8mm video tape cassettes (Sony P6-120MP, NTSC).
These small cassettes, when used with the EXABYTE
8200 streaming tape sub-system, can hold up to 2.2
Gigabytes of digital information each. All WSI data
manipulations conducted at MPL are built around this
small compact tape system under the control of an IBM/
AT class microcomputer. All software is MS-DOS
compatible, and is generally written in either FORTRAN
or C programming language.
The basic hardware configuration required for WSI
image processing is essentially the same as that required
for the initial acquisition. That is, a system able to read
the 8mm data tapes, an image processing system able to
handle the 512 X 480 pixel image format, and a host
computer suitable for overall control. At MPL, the data
acquisition system shown in Fig. 2-7 is also appropriate
for all subsequent image processing tasks. For bulk
processing, however, the basic computer unit is supple-
mented with an Eighteen Eight Labs PL 1250 floating
The WSI data base was acquired primarily in support
of the SDI ground-based laser program. A variety of
models predict joint multi-site probabilities of cloud free
lines of sight (CFLOS) for earth to space, and cloud free
arcs (CFARC). These joint probabilities depend on a
variety of intermediate level relationships, such as CFLOS
and CFARC as a function of sky cover and zenith angle,
site-to-site correlation as a function of separation dis-
tance, and temporal relations such as recurrence and
persistence probabilities.
The WSI data base is in many ways an ideal set for
model evaluation. The model input, i.e. sky cover, is
measured directly. The various intermediate level results
mentioned above may be computed directly from the
digitized data base, and the final multi-site joint prob-
abilities may be computed directly from the data. As a
result, WSI can provide the data base for testing existing
models, development of new models, and documentation
of a mini-climatology of CFLOS and CFARC at specific
sites of interest.
There are other applications within the user commu-
nity for using these data. For example, there is interest
in creating a climatology of optically thin cloud cover,
eventually as a function of optical depth. Additionally,
the fact that absolute radiometric calibrations are acquired
for the instruments means that a data base of calibrated
radiance could be generated from the data, for back-
ground clutter and other applications.
3.2 Current Extent of the Data Base
The Whole Sky Imager (WSI) data network, while
fully implemented, consisted of seven independently
operating, fully automatic electronic camera systems.
Each system collected whole sky imagery for twelve
hours a day, seven days a week. Each resultant weekly
data tape currently in archive contains up to 1200
megabytes of image data, plus around 200 megabytes of
housekeeping space. Thus, at the end of the data collec-
3

IMAGE & ANALYSIS SYSTEM
HARDWARE BLOCK DIAGRAM E/O SYSTEM 5
GE 2710
SOLID STATE
VIDEO
CAMERA
SONY PVM 1271Q
MONITOR
TMI COMPUTER
(1BMAT CLONE)
AUTOMATIC
EQUATORIAL
SOLAR
OCCULTOR
ASSY.
VIDEO
IMAGE
PROCESSING
SUB - SYSTEM
(ITI FG 100)
ARCHIVAL
I/O
SUB - SYSTEM
(SEAGATE
65 Mbyte H.D.)
За
REMOTE
CONTROLLED
IRIS
ASSY.
ANALOG
EXABYTE EXB - 8200
2.2 Gbyte 8 mm CATRIDGE TAPE SYSTEM
.
PUNIM
..10
ad
!
REMOTE
CONTROLLED
OPTICAL
FILTER
ASSY.
ACCESSORY
CONTROL PANEL
STOWED KEYBOARD
EXTERIOR SENSOR
INSTALLATION
INTERIOR CONTROLLER
INSTALLATION
Figure 2-7
pr.53. Sensor Inst/Janet
tion interval 31 Dec 90, the WSI cloud data base con-
tained approximately 900 Gigabytes of raw image data
and about 675 Gigabytes of processed data. The data
quantities are substantial, and require carefully con-
ceived and efficiently executed processing procedures to
insure optimum downstream utilization.
The WSI cloud data base summarized in Table 3.1
presently resides in the custody of the Marine Physical
Laboratory (MPL), Scripps Institution of Oceanography,
San Diego, CA, 92093. As of 1 Jan 91, imagery repre-
senting 4614 days, geographically distributed as shown
in Table 3.1 and Fig. 3-1 have been collected and returned
to MPL for processing and analysis. A reasonable
estimate is that these raw data reside on approximately
900 data tapes.
"WSI BASICIMAGEPROCESSING FLOW CHART".
This conceptual game plan is implemented through a
series of proprietary software routines designed to impose
strict quality and calibration controls upon the raw data,
while simultaneously reducing the number of images
required for insertion into the optimized cloud/no cloud
decision processes. Carried to its nominal conclusion,
this processing plan reduces the basic imagery (that
created by each camera in the system) to about one fourth
of its original volume at the composite ratio stage and
then, if desired, by an additional increment depending
upon the output format specified by the user of the
derived products. Currently (Jan 1991) the optimum
cloud/no cloud images are being created at the same eight
bit resolution as are the composite ratio images. Whereas
further data compaction might prove attractive for later
data transfers, the additional software development
deemed necessary has not seemed justified at this time.
4.1 Data Processing Procedural Sequence
With the basic data tapes in hand, the processing
hardware and software on-line, and the data base man-
agement system implemented, one can address the
stepwise manipulation of all component elements from
raw tape to delivered end product. In the following
paragraphs this stepwise process will be described on the
basis of a single data tapes sequential manipulation, and
from this unit tape estimate, the overall program time/
cost estimates will be produced.
The preparation of these 900 raw data tapes plus their
appropriate output products with the procedural docu-
mentation deemed necessary for subsequent usages is
essentially the task outlined in this note.
It should be noted that these data quantities do not
include several relatively small case study sets collected
by the portable system. The data listed in Table 1 and
discussed in this note are only those associated with the
multi-station network sponsored under Air Force con-
tract F19628-88-C-0005.
4.0 STANDARD DATA PROCESSING
PROCEDURES
U
L
...
The general concept guiding the post-acquisition pro-
cessing of the WSI data base is illustrated in Fig. 4-1,
In assessing the following estimates of the processing
times associated with each procedural sequence, one
Table 3.1

WSI DATA TAPE STATUS AS OF 31 JANUARY 1991
C-STA HELSTF
30 MAR 88 | 29 MAR 88
KIRTLAND CHINA LAKE MALMSTROM MALABAR
17 MAY 88 | 23 JUN 88 29 AUG 881 18 NOV 88
COLUMBIA
9 FEB 89
TOTAL DAYS IN FIELD
995
1008
953
897
478
735
691
763
819
749
641
406
602
592
TOTAL DAYS OF DATA
TOTAL FULL DATA DAYS
(12 HR)
C
652
623
607
453
332
515
561
111
196
142
188
74
87
31
OTAL PARTIAL DATA DAYS
TOTAL DAYS DOWN
(NO DATA)
E
232
189
204
256
78
133
99
DATE WITHDRAWN
FROM SERVICE
19 Dec 90
31 Dec 90 25 Dec 901 6 Dec 90
| 20 Dec 89 | 19 Nov 90 | 31 Dec 90
REG
.
A=B+E
B = C + D
WWE WWW.

WSI CUMULATIVE NUMBER OF DATA DAYS
KKKKT
E 1000
-
4а
Is
o las Bonisimin
SITE
SITE 2
SITE 34
SITE A
SITE SI
SITE 6
SITE 7
SITE 1
SITE 2
SITE 3
SITE 4
SITE 5
SITE 6
SITE 7
Fig. 3-1

w
w
ww
...
..
w
ith
..
.
.
.
BASIC IMAGERY
COARCTED IMAGERY
COMPOSTE AATIO
DELMERABLE DATABASE
CALIBRATION
FUNCTIONS
AADIANCE
CONVERSIONS
CLOVONO CLOUD
DECSION
ALCOANHIMS
CALB
QUE
MAGE
ALVE
MAGE
512 * $1211
RADIOWE TAK
I NE AANY
C
ELVERED
RATOS
RADIOMETAIC
SENSITNIY
CAD
UP SUN
RED
MAGE
512 151216
RED
DEANED
PRODUCTS
MAGE
DOWN SUN
OPTICA
DBTOATIONS
MAGE AATIO
COMPUTATIONS
MEMA HORVON
SPATUL
DISTABUTIONS
COMPOSITE
BLUE/RED RATIO
MACE
OPTIMUM
CLOUDNO A QUO
MACE
TWLIGHT
FELD OF VEW
DEF NITION
TEMPORAL
TOISTRATIONS
QUE NO
MAGE
512131211
CALE
(O.VE)
MAGE
DAWN
SENSOR CHP
UNFORMITY
STATISTICAL
PARAMETERS
7 AS ROD
PREL SELECTIONS
FOR
OPTIMUMIZED
COMPOSITE
.
(PUEYRED)
MATOS
..
.....
REOSTRATION
ADUSTMENTS
REDNO
MADE
312151211
CALB
(RED)
MADE
FLUX CONTAC
THRESHOLOS
LJ
_
J
Fig. 4-1. WSI Basic Image Processing Flow Chart
site host personnel with widely varying degrees of tech-
nical background and program related interests. Not all
Fig. 4-2
Raw Imagery to Decision Imagery Conversions
ono sto ono wook
RAW
TAPE
TAPEQC
TAPEOC
OUTPUT
FILES
T.O.F.
BACK-UPS
RAW
TAPE
EXCOPY
PAW
DATA
BACK-UP
(ARCHIVE)
5
T.O.F.
(BACK-UPS
.
CHECLAN
-3
TAPRAT
INPUT
FILES
meseca
(INTERACTIVE)
RAW
TAPE
TAPRAT
INPUT
FILES
TAPRAT
should be aware that throughput times are often heavily
influenced by different machine/software interfaceseven
when determined on systems containing nominally
"identical" hardware and code versions. With this caveat
in mind, the estimates included herein and indicated on
Fig. 4-2, were derived from actual full length processing
episodes running in the hardware/soſlware conſiguration
listed in Table 4-1.
4.2 Data Processing Run Time Estimates
Processing the complete stepwise procedure required
to convert a field tape containing seven days of raw
unedited cloud imagery into a fully processed tape con-
taining the same seven days of data in the form of color
coded cloud-no cloud imagery is illustrated in Fig. 4-2,
and the run-time summary for both machine time and
supporting analyst time is shown in Tablc 4.2.
In estimating the overall processing task, as outlined
in Fig. 4-2 & Tablc 4.2, it is important to note that the
totals shown in Table 4.2 are really for best effort data
"processing" only. They do not include the very substan-
tial effort which is heavily related to "housekeeping and
oversight", but which is more speciſically oriented to-
ward on-line data "interpretation and analysis". Nor do
they include the more stylized but highly important
implementation and maintenance of the required Data
Base Management functions.
The Table 4.2 comment related to "clean" runs is a
comment ignored at extreme peril. One must keep in
mind that all of WSI field systems were monitored by on-

PATIO
TAPE
110
LOG FILE
DIAG FILE
EXCOPY
AATIO
TAPE
1/10
RATIO
BACK UP
(OPTIONAL
ARCHIVE)
2
LOG FILE
DATA BASE
SUMMARY
0.8.
SUMMARY
AATIOS
DIAG. FILE
AATIO
TAPE
1/10
CLDOEC
CLODEC
TAPE
110
CLDOEC
OUTPUT
FILES
EXCOPY
(ARCHIVE)
CLDOEC
TAPE
1/10
ADDEC
BACK-UP
1/10
CLDOEC
OUTPUT
FILES
to SUMMA
DATA BASE
SUMMARY
D.B.
SUMMARY
DECISION
Separate runs required for each one minute or ten minuto Imago sot.
wwwwwwwW
A
-
INKWIN-WWM-
WIN-
"
Table 4.1
WSI Standard Data Processing Configurations
HARDWARE
SOFTWARE
received the imposition of the WSI system into their daily
routines with equal exuberance. The vagarics of local
wcather extremes, power fluctuations, hardware ſaults
and conſlicting on-site prioritics all contribuicd to
sometimes substantial variations in the normally full
automatic mode that the WSI altcmpted to maintain.
With this "real-world" context in mind, it is casy lo
visualize the nature of the on-line data interpretation and
analysis that must accompany the more formalized data
"processing" outlined in Fig. 4-2.

TMI Model 2001A Computer
Propriclary Softwarc Programs
1. 80386 CPU
2. 80387 Co Processor
| 3. PL 1250 Array Processor
4. ASC-88 SCSI Interface
5. EXABYTE 8200, 2 ca
I w/ 4 $25 PROM
6. Seagate 60 MB Hard Disc
| 1. TAPEQC, Version xxx
12. EXCOPY, Version xxx
3. TAPRAT, Version xxx
4. CLDDEC, Version xxx
5. DBSUM, Version
(In development)
.
.
.
.....
Table 4.2
Raw to Decision Conversions
Run Time Summary
Machinc Hours
Analyst Hours
Primary File Copy Total Housekecping &Oversight
1
Sequence No.
(From Fig. 4-2)
The primary tools employed by the analyst to diag-
nose and sanitize those raw data tapes which ſail either
their initial TAPEQC runs, or their subscquent TAPRAT
runs, are the regularly produced TAPEQC output files,
the Local Apparent Noon (LAN) image files, and the raw
data tapes themselves, all in concert with the analysts
highly honed understanding of the WSI hardware and
software operating regimes. These diagnostic skills are
not readily transſcrable and they are absolutcly csscnlial
to the orderly and efficient data ſlow shown in Fig. 4-2.
Some estimate of the additional machine time re-
quired to support the "interpretation and analysis" (1/A)
function plus the attendant requirement for data re-runs
should be provided as part of the overall task deſinition.
As a ſirst approximation, one finds about 7% of the data
designated as special handling. During the ſirst quarter
of calendar 1990, an estimated 10% of the accumulated
data runs were repeated due to specific 1/A decisions and
another 10% were re-run due to miscellaneous hardware
pcculiarities, primarily EXABYTE incompatibilitics.
Thus one might assume a worst casc requirement for up
10 27% of miscellancous re-runs, however as "production
line" processing settles down, a 10 to 15% re-runestimate
scems more reasonable.
4.3 Data Processing Status

0.5
2.5
0.5+0.5
0.5+0.5
ooo Baum WN-
.
Pm RM
0.5
...
6+6
0.5+0.5
.27.2
0.5+0.5
2
Totals 1+10
6.2 64.2
1or 10
* Separate runs roquired for each one minute or ten minute image set.
Times indicated are based upon "clcan" runs without appreciable software or
hardware glitches.
-
4.8
..
.
As noted in Section 3.2, there are approximately 900
raw data tapes currently in archive status, representing
over 4600 separate data days. Some of these data tapes
have been processed through all of the sequences illus-
trated in Fig. 4--2, however the majority have been
through steps one and two only. An illustration of the
data base status as of 30 Dec. 90 is shown in Fig. 4-3. A
slightly more detailed representation broken down by
major software program is shown in Table 4.3.
5.0 ALTERNATIVE PRODUCTS
Table 4.3
Processed Data Tape Summary
Raw
Major Software Programs
Data
Tapes TAPE QC CHECKLAN TAPRAT

Data
Station
CLDDEC
11 10
|
10
C-Sta
70
|
70
|
70
Ilcisir
......
..
Kinland
China Lake
Malmstrom
Malabar
Columbia
Toials
All preceding comments have been directed toward
the "standard" data processing mode as illustrated in Fig.
4-1. The output product for this standard sequence is a
database containing derived cloud/no cloudimages based
.
Chutne
"

1
FEB MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT. NOV. DEC.
I
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG. SEP.
OCT. NOV. DEC.
CSTA
CSTA
HELSTF
HELSTF
KIRTLAND
KIRTLAND
CHINA
LAKE
CHINA
LAKE
MALMSTROM
MALM-
STROM
MALABAR
MALABAR
OLUMBIA
1988
1989
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT.
NOV.
DEC.
CSTA
HELSTF
WSI DATA STATUS &
PROCESSING SUMMARY
= GOOD DATA
KIRTLAND
SPECIAL HANDLING REQUIRED
- SYSTEM DOWN
= INSTALLATIONMAINTENANCE
CHINA
LAKE
= RATIO
= CLOUD DECISION
MALM-
STROM
Figure 4-3
MALABAR
COLUMBIA
1990
of the form shown in Fig. 5-1, with programs ABSRAD
and SELECT representing the necessary new code.
upon a single threshold decision algorithm (APR '90).
There are other output products which have been dis-
cussed at different times during the data acquisition
interval that should be commented upon as viable op-
tions.
to the best pornes ales R.AD

Fig. 5-1
Raw Imagery to Callbrated Radiance Data
ABS RAD
ABS
RADIANCE)
TAPE
1/10
1
SELECT
The first alternative involves the use of improved
decision algorithms to up-grade the cloud/no cloud data
base. In general this up-grade can be exercised at
anytime and as often as desired, once the composite ratio
image library has bcenicstablished. Re-running TAPRAT
output lapes through a revision of CLDDEC is straight
forward and relatively inexpensive, as can be deduced
from Table 4.2.
(4.*)a
RAW
TAPE
TAPAAT
INPUT
FILES
TAPRAT
RATIO
TAPE
1/10
LOG FILE
DIAG FILE
|(5.*)a
ABS RADI
EXCOPY
RADIANCE
BACK-UP
1/10
(OPTIONAL
ARCHIVE)
TAPE
1/10
5.1 Upgrade CLDDEC
__
wie
.
LOG FILE
DATA BASE
SUMMARY
D.B.
SUMMARY
RADIANCE
DIAG. FILE
0
Separate runs required for each one minute or ten minute image set.
.
X
A
Exccuting an up-grade to the cloud/no cloud imagery
via an improved cloud decision algorithm implies re-
running steps 7, 8 & 9 as shown in Fig. 4-2. The
associated machine and analyst hours necessary for this
task are listed in Table 4.2, and represent 16.6 machine-
hours and 2.5 analyst hours per ratio tape. Assuming
ratio tapes remain at 1:4 with raw data tapes, then about
225 tapes would require this partial re-processing. Some
tape compaction is possible through a judicious applica-
tion of EXCOPY, but it is not currently an automatic
option. Thus, the re-run of 225 tapes through CLDDEC,
etc., represents a task of 3,700 machine-hours (928
system-hours) and 560 analyst hours, or 3.2 analyst-mo
and 2.5 system-mo.
The second alternative involves the modification of
the Fig. 4-2 sequence to generate calibrated radiance
imagcry, citcr in addition to or in licu of the cloud/no
cloud imagery. This sequence modification is not trivial
since the calibrated radiance imagery is not normally
held intact in image memory, but for maximum compu-
tational throughput is used piecemeal and not retained
once the ratio computations are complete. An estimated
cost for implementing this alternative can also be de-
duced from the estimates in Fig. 4-2.
5.2 Create Calibrated Radiance Library
Since the absolute radiance values which are desired
as output reside temporarily within the existing sequence,
and their accumulation should be reasonably simple, the
machine time necessary to execute the ABSRAD task
should represent a relatively short interval. A reasonable
first approximation for the ABSRAD execution time is
its being equivalent to or less than the CLDDEC runs. To
the degree with which this approximation is truc, then the
incremental cost of executing the sequence shown in Fig.
5-1 can be estimated.
Assuming a clcan inscrion, it is reasonablc to assume
approximately 3 man-mo lo code, test and dcbug the
ABSRAD & SELECT software. Housekeeping and
Oversight (H/O) analyst hours required for processing
should not exceed the TAPRAT rate, nor should the I/A
time. Thus an estimate of around 4 analyst-hours per
radiance tape, about half the TAPRAT rate, scems rea-
sonable.
Rerunning the nominal 100 raw data tapes through the
SELECT/ABSRAD sequence represents an incremental
increase of 600 machine hours and 400 analyst-hours
(approximately 1.0 system-mo. and 2.3 analyst-mo.). An
estimated incremental cost for implementing this alter-
native processing sequence is readily determinable.
6.0 WSI DATA EVALUATION
Modifying the processing sequence shown in Fig.4-2
to produce output lapes of calibrated radiance imagery
has not yet been implemented, but does not represent a
major programming effort provided one makes a clean
interrupt at the point of insertion within the software
program TAPRAT. There will be however, changes to
the logical flow within the program which may result in
substantial increases in processing times. In any case,
the creation of a calibrated radiance library would
procedurally involve the re-running of modified steps 4,
5,& 6 shown in Fig. 4-2. These modifications would be
1
Comparison with Observer
Much of the WSI data from the Columbia site has been
processed using a preliminary (fixed threshold) algo-
rithm, in order to give a preliminary assessment of the

accuracy of these techniques. Fig. 6-1 shows the cloud
cover distribution from 7 months of WSI data, compared
with the values of total sky cover reported on the National
Weather Service Form 10's. The WSI values are from
the one minute image at the reported time of the weather
observation. This plot is for the six hours surrounding
local apparent noon. The comparison between WSI and
observer is in general quite good. The cloud algorithm
identifies some clear cases as 1/10 cover, but in all other
cloud categories the match in frequency of observance is
excellent.
6.2 Cloud Cover Temporal Dynamics Illustration
Although the observer to WSI comparison is gener-
ally good, the WSI data can show much more variance
than the observer data, simply due to the limited temporal
frequency of the observer values. One particularly dy-
namic day, 14 April at Columbia, Mo., is illustrated in
Fig. 6-3. In Fig. 6-3, which shows the cloud cover
COLUMBIN, MO 14 APEL 1989
OUSERWER
Terme SKY covee cerurteta, MO
APN RUST NUNCT 14
TOTAL CLOUD COVER (E)
BU
RELAIME FROUKY (N)
TIME (6411
Fig. 6-3. Total Cloud Cover Time series, WSI and Observer
o
011
13 14 15
SKY COVE# AruNT (PENTRS
Fig. 6-1. Distribution of Total Cloud Cover
determinations, WSI and Weather Observer.
Another indication of data quality is a direct case-by-
case comparison between WSI and observer. For Fig.
6-2, the difference between WSI and observer has been
computed for each case. That is, a WSI value of 7/10 and
observer value of 5/10 would be a difference of 2 catego-
ries. The distribution of category differences is shown in
Fig. 6-2. The majority of the cases show a category
difference of 0. The average difference is less than half
a category, i.e. much less than 1/10 cloud cover.
COMUNIKA
O 6
wo STOW1**
determinations for both the WSI and the observer, the
observer values are consistent with the WSI, but they
certainly do not show the true variability, due to their
limited temporal frequency. For example, at 1350 and
1450, both WSI and observer show approximately 20%
cloud cover, but during the intervening hour the cloud
cover increased to nearly 80%. The cloud images at the
middle and end of this hour are shown in Figs. 6-4 and
6-5. The fastest rate of change during this hour occurred
at between 1400 and 1413, when the cloud cover changed
from 20 to 74%.
SKY COW 766
1:01
LATIVE TROUENCY
COUD COVER CATEGORY DUFRERE
«
W
SEWE)
Fig. 6-2. Total Cloud Cover determination difference for
WSI minus observer.
Fig. 6-4. WSI Cloud Cover Image at 1420, from
above time series.

WWWWWWWWWWWWW
Fig. 6-5. WSI Cloud Cover Image at 1450, from
above time series.
7.0 CONCLUSION
With the development of a family of digital imaging
systems, it has been possible to support a number of
applications requiring monitoring of the cloud environ-
ment. The Whole Sky Imager has been monitoring
specific sites for post analysis and evaluation of cloud
fields, with high temporal and spatial resolution.
The WSI has been used specifically to gather a data
base of over a million images for assessment of cloud
fields. The data consists of cloud decision images,
containing an assessment of the cloud condition in each
direction with 1/3 degree spatial resolution, and one
minute temporal resolution. These data are currently
undergoing processing and quality evaluation. The
preliminary results appear quite good, in comparison
with standard meteorological observations.
8.0 ACKNOWLEDGEMENTS
This work was sponsored by Geophysics Lab, Air
Force Systems Command, under contract #F19628-88-
K-0005. Our thanks to Mr. Donald Grantham and Dr.
William Snow of GL for their guidance. The authors
would also like to recognize the outstanding efforts of
our colleagues at Marine Physical Laboratory: Wayne
Hering, Monette Karr, Gene Zawadski, Jack Varah,
Harry Sprink, Melissa Ciandro, Tim Romedy, John
Malo, and Peter Pak, for technical support, and Phil
Rapp and Carole Robb for publications support.
9.0 REFERENCES & BIBLIOGRAPHY
Duntley, S. Q., R. W. Johnson, J. I. Gordon, and A. R.
Boileau, (1970). Airborne Measurements of Optical
Atmospheric Properties at Night, University of Cali-
fornia, San Diego, Scripps Institution of Oceanogra-
phy, Visibility Laboratory, SIO Ref. 70-7, AFCRL-
70-0137, NTIS No. AD 870 734.
Hering, W. S. and R. W. Johnson, (1985). The FASCAT
Model Performance Under Fractional Cloud Condi-
tions and Related Studies, University of California,
San Diego, Scripps Institution of Oceanography, Visi-
bility Laboratory, SIO Ref. 85-7, AFGL-TR-84-0168,
NTIS No. AOA 085 451.
Hering, W. S., (1989). Evaluation of Stochastic Models
for Estimating the Persistence Probability of Cloud-
Free Lines-of-Sight, University of California, San Di-
ego, Scripps Institution of Oceanography, Marine
Physical Laboratory, GL-TR-89-0275.
Johnson, R. W., T. L. Koehler, and J. E. Shields, (1988).
A Multi-Station Set of Whole Sky Imagers and å
Preliminary Assessment of the Emerging Database,
University of California, San Diego, Scripps Institu-
tion of Oceanography, Marine Physical Laboratory,
Atmospheric Optical Systems Tech Note No. 210.
Johnson, R. W., W. S. Hering, and J. E. Shields, (1989).
Automated Visibility and Cloud Cover Measurements
with a Solid-State Imaging System, University of
California, San Diego, Scripps Institution of Ocean-
ography, Marine Physical Laboratory, SIO 89-7, GL-
| TR-89-0061.
Karr, M. E., and J. E. Shields, (1989). Whole Sky Imager
Management of Raw Database, University of Cali-
fornia, San Diego, Scripps Institution of Oceanogra-
phy, Marine Physical Laboratory, Atmospheric Optical
Systems Tech Note No. 211.
Shields, J. E., T. L. Koehler, M. E. Karr and R. W.
Johnson, (1990). Automated Cloud Cover & Visibility
Systems for Real Time Applications, University of
California, San Diego, Scripps Institution of Ocean-
ography, Marine Physical Laboratory, Atmospheric
Optical Systems Tech Note No. 217.
g