In 1994, 16 stream transects were established in the McMurdo Dry Valleys of Antarctica 63 beginning a long term data set characterizing microbial communities and channel geometry. The 64 transects were established to record microbial mat dynamics and stream geomorphology. To accomplish this, the transects were surveyed for points of interest outside and inside the stream channel. Beginning in 2010 the microbial surveys received ground based LiDAR support. This allowed for greater resolution in mapping and analyzing stream morphology than traditional surveying methods. Using surveyed microbial mats as an indicator of a location in time, a history of channel elevations was created for 7 transects. Here, you will find the massive long term data for the geomorphology of seven of the streams, along with detailed references to the methodology and results and other findings published in Steven Crisp thesis dissertation.
The purpose of this study was to overlap the traditional methods of surveying individual points of interest with a data cloud 69 representing the entire stream transect to be able to continue the microbial study into the future unabated.
Methods for digitizing contour plots, adding Lidar data, and calculating DEM’s of difference
Note: If something doesn’t work. Save Arc. Close arc. Open arc. Magic!
I. Scanning Alger Contour plots into ARC
a. Adding map document to Arc found at http://courses.umass.edu/nrc592g-cschweik/pdfs/Georeferencing_a_map.pdf
b. Copy image (Contour plot from Alger) into paint.net and save as XX_Alger.jpg
c. Start ArcMap
i. Customize è extensions è check “3d Analyst, ArcScan, and Spatial Analys
ii. Customize è Toolbars è Add ArcScan, Georeferencing , and Editor
iii. File è Map document properties è check “Store relative pathnames to data sources”
iv. Catalog è Connect to folder è “Thesis”
d. Setting jpg properties and adding to map
i. Folder Connections è .jpg file è right click è properties è Spatial Reference è Edit è WGS 1984 UTM 58 S è Appl
ii. Add Data è jpg file è Build pyramids (Yes)
e. Georeferencing Contour map
i. Click “Add control points” on georeferencing toolbar
ii. Zoom in to BM1
iii. Click once on BM1 then again to turn crosshair red from green
iv. Repeat for all BM’s
v. Once all BM’s are marked, open excel file “1993_Survey_All_Transects” and note lat/long of all BM’s
vi. Open “Link Table”(Sometimes located just off edge of screen…) è Replace X_Map and Y_Map with BM’s lat/long è Zoom to full extent to go to true map location
vii. In georeferencing toolbar, click “Rectify” è Change save location to proper folder è Save saves as XX_Alger1.tiff file with “1” added to end of name
viii. Close Map Doc è start blank map è Click “+” (Expand tiff file) button to new tiff file è Add 1 layer (Band_1) of rectified tiff to new map
ix. In Table of Contents, right click Tiff file è Properties è Symbology è Classified è Classes = “2” è “OK”
II. Creating Vector data from contour lines in Alger Scans
a. Information found at http://www.fas.harvard.edu/~chinaarch/Downloads/gis/L05_creatingvector/L05_creatingvector%20English.pdf
b. Creating a new shapefile in arc catalog
i. In Catalog, right click folder that contour plot is in è New è Shapefile
1. Name with NO SPACES similar to “XX_Shp.shp”
2. Feature Type = “PolyLine”
3. Coordinate System = “58 S”
ii. Adding Attributes
1. Right Click new shapefile in Catalog! è Properties è “Fields” Tab è Below ID in Field Name type “Elevation” è under Data Type select “Double” è “OK”
c. Creating Vectors
i. In Editing Toolbar select “Start Editing”, Then select the Shapefile in “Create Features”, and then “line”
ii. In ArcScan toolbar select “Raster Snapping Options” button è Check for correct values (What are they? 20,5,200,0) è “OK” Make sure arcscan is up and other features to make vectorization work
iii. Under the “Vectorization” Menu, select “Vectorization Settings” è “Styles” è “Contours” è “OK”
iv. Next to the Vectorization Menu, select “Vectorization Trace” Tool.
v. Click on beginning of Contour Line, and allow arrow to trace line automatically
1. Sometimes at intersections or other unnatural features, the line stops. “Jump” Over the break with “s” and continue on with line
2. Double Click or press F2 to end Vector trace
3. Arc loves to not work after crossing a line. It jumps giant gaps to connect with completely unrelated lines although it should follow the line indicated. Need to contact Arc about this, or just finish the contour manually.
4. If there are large gaps in the contour lines, manually change the classification scheme to account for gray areas to be black and not white.
vi. ArcScan allows for automation of contours, but I don’t believe this works very well. Instructions in website or pdf above
vii. Once line is created, go into Attribute Table of shapefile and add elevation of contour line just created
1. Check BM elevation from Alger
2. Check BM elevation from UNAVCO
3. +/- difference from the two for each contour elevation
viii. Repeat for all contour lines
ix. “Stop Editing
x. Save Map Frequently as ex. “XX.mxd”
III. Topo to raster 3D analyst for now? Spatial does stream polyline better(Not using this anymore)
a. Input feature data = XX_Shp.shp
b. Field = Elevation
c. Rename and save to proper folder! “XX_T2R”
d. Cell size=? Make everything 0.1 (meters) but takes forever with Lidar
e. After finishing, can change the symbology to full 32 layers instead of ~9.
1. Need to figure out how to label Orange, Green, and Black mats so can tell differences between mats and seasons.
f. Water Surface Elevations. Maybe?
h. Other important features
IV. Cropping out extra raster (Not using this either anymore)
a. Need to crop out extra raster data because is not correct or accurate
i. Make new shapefile (XX_Clp.shp)
1. Polygon, and same coordinate system
ii. Start Editing, select XX_Clp
iii. Trace perimeter of contour shapefile. Make sure end snapping is selected.
iv. Clip(Data Management)
1. Select Raster to clip è XX_T2R
2. Output extent è XX_Clp
3. Select “Use input features…”
4. Rename output è XX_T2R_Clp
5. May need to change symbology from stretched to classified. This asks to compute histogram. Select yes. Then import data from XX_T2R for same color scheme.
V. Importing LiDAR data into ARC
a. If Lidar data is single file continue to b. Otherwise follow below.
i. Open up a new command window in dos. Click start and type cmd
ii. Navigate to folder containing lidar sets using the command “cd foldername” to go up one folder at a time or “cd..” to go back a folder. Ex. C:\Users\Crisp\Desktop\Thesis\Antarctic Scans\Lidar\F2
iii. Type “copy /b filename1 + filename2 + filename3 outfilename
iv. Concatenated file should appear in a few seconds!
b. In Arc, Under “File” è Add data è Add XY data (This is very important XY data!!!)
i. Select LiDAR set
ii. Choose corresponding X, Y, and Z columns for data
iii. Update Coordinate system to 58S
iv. Hit OK
1. Once the addition is finished, it will bog down ARC adding each point. Unselect the Lidar data in the table of contents and proceed to point to raster below
c. Point to Raster in conversion tools
i. This is going to take a while so be prepared
ii. Choose newly imported XY data set
iii. Value field is Z or elevation
iv. Save file as XX_P2R_YYYY Unfortunately Arc is omnipotent and gets angry when there are more than 13 characters. Assumption is that cellsize (Below) is 0.1m
v. Cellsize = 0.1
1. This means each cell with be 0.1 m wide. You can change this to a smaller dimension, but it will take exponentially longer.
vi. You can get into detail with cell assignment and priority fields, but typically leave them at default
vii. Wait for 35-45 hours (min) for this to finish.
viii. Once completed, change symbology to from stretched to classified
1. Symbology adoption however has problems.
2. Ex. Lidar has range of 19-31 meters. Adopted contour plot has range of 25-28.
3. Therefore everything on Lidar below 25 or above 28 is same color.
4. Fix? Find the best fit scan/plot and correlate everything else to that piece
d. Patching holes in raster and adding hillshade
i. “Raster Calculator” in Spatial Analyst
1. Enter code è Con(IsNull("Input_Raster"),FocalStatistics("Input_Raster",NbrRectangle(3, 3),'MEAN'),"Input_Raster") è Output Raster = “XX_Patch” è OK
2. The input raster is the lidar DEM that has holes in it either due to water or shadows from rocks or poor TLS placement
3. The “3” is the width and height of the patching
4. Explanation: The above code finds cells within the DEM that are NoData and takes a 3X3 matrix surrounding it, computes the mean, and then outputs that elevation value
ii. “Hillshade” Spatial Analyst
1. Input Raster = XX_Patch è Output Raster = XX_YYYY_HS è azimuth altitude and z are optional to change è OK
2. Need grayscale
iii. Represent stream within channel at certain flows?
iv. Make a model builder!
VI. Adding LAS data to Arc Scene
i. Open Txt2Las in the lastools/bin folder
ii. Click Browse è \.. to go back folder. Search for RGB lidar file you want
iii. Output è Proper directory è Name file XX_YYYY_LAS
iv. More attributes è Select R, G, and B
v. CHANGE FORMAT FROM LAZ TO LAS!!! Have no idea why this isn’t automatic
vi. Hit run! è takes ~30 seconds.
b. Importing LAS file
i. Open new arcscene map and save as XX.sxd
ii. Navigate to folder being used for stream in folder connections
iii. Right click stream folder è new è LAS Dataset è name “XX.lasd”
iv. Right click XX.lasd è Properties è XY coordinate system tab = 58S è LAS Files tab è Add files è select newly created las file from lastools è OK
v. Drag XX.lasd into map
vi. In the LAS Dataset Layer toolbar select the symbolize button (Button with bunch of dots) and select RGB.
vii. Now have lidar map to fly through!
VII. GCD – Geomorphic Change Detection
a. Download and Install
i. Download and install GCD 6 from http://gcd.joewheaton.org/downloads
ii. Go to Arc è Customize è Add in manager and GCD should automatically be added
iii. In toolbars select GCD and pin it to the toolbar
b. Building DEMS
VIII. Adding XY Algae Data to Alger Scans
a. Algae Data
1. Into .csv’s
a. Data should already be organized in one large spreadsheet based off of flow season
b. Create 1 csv file for 1 stream for 1 season
i. Ex. Bowles_1213.csv, Bowles_1314.csv…
ii. .csv files should have 4 columns
3. Orthometric height
ii. Adding XY data
1. Find .csv file è Create feature class è From XY table
2. Change x and y and coordinate system
3. Save as default name
4. This creates a shapefile within the stream folder and not the default Arc folder which might be better for editing and finding etc.
5. Add new shp file to arc map
b. Stream Boundary
i. Create new polygon shapefile called “Stream_Boundary”
ii. Start editing
iii. Follow stream path and save edits
c. Contour Polyline to point
i. Feature Vertices to Points
1. Input Features = “XX_Shp”
2. Output = “Contour_Points”
IX. Point differencing from base lidar
a. G = 1, B = 2, O = 3, R = 4
b. Extract Values to points
i. Input point features “XYXX_YYYY”
ii. Input Raster “Patch”
1. If not doing patch, the NoData cells indicate water. Can be used for algae analysis later on in different study
iii. Name “YYYY_E2P”
1. YYYY = year
2. E2P = Extract to points
iv. Interpolate points Check
i. Input features = YYYY_E2P
ii. Clip Features = Stream_Boundary
iii. Output = YYYY_Clip
i. Input features = YYYY_E2P
ii. Erase Features = Stream_Boundary
iii. Output = YYYY_Erase
e. Export feature attribute to ASCII – Find in toolboxes not custom box
i. Input feature = YYYY_clip/erase
ii. Value field = Orthometri, RASTERVALU, ( + Point_Desc for Clp file!)
iii. Delimiter = space
iv. Output = YYYY_Ers/Clp
v. Field names = Unchecked!
f. Contour Clip and Erase
i. Perform same as above except export the contour season as 9293 so it plots next to 9394 or the season that it was created. Maybe figure a better way to plot this?
Packaged by Crisp, DEIMS preps by Inigo, LTER NIS package DEIMS.