Geospatial Concepts: Recent Episodes
mapscaping.com
In this podcast episode, Jennings Anderson, a research scientist at Meta, discusses the Overture Maps Foundation, a downstream product of OpenStreetMap.
He explains his background in open map data and his interest in studying collaboration within the OpenStreetMap community.
Jennings then dives into the Daylight Distribution, an open data product produced by Meta, and how it combines building data sets from various sources into one unified theme.
Jennings emphasizes the importance of a stable ID system within the Overture Maps Foundation and the potential for easy conflation and integration of third-party data.
Jennings also explains the relationship between OpenStreetMap and Overture Maps, highlighting how they complement each other.
Relevant podcast episodes
OpenStreetMap Is A Community Of Communities
Cloud Native Geospatial
Cloud Optimized Point Clouds
The Rapid Editor
With regards to accessing Overture Map data, you might find this YouTube video helpful
https://youtu.be/fZj6kTwXN1U?feature=shared
Just in case you are interested in the Google building footprints here is a link to that :)
https://sites.research.google/open-buildings/
100 billion Points Every Day
100 billion is a very large number, let's say that I gave you a spreadsheet with 100 billion rows in it, each row consisted of five columns Latitude, Longitude, Device ID, A Timestamp, and a column telling the name of the data provider
What would you do with that?
How would you clean it? Make sense of it? Extract value from it? What would people use it for? And how would you do this in a way that could be systematized?
FourSquare does this every day with the help of something they call a movement engine.
To help understand more about how they do this I have invited Gabriel Durkin the director of data science on the podcast. This is the last in a series of episodes I have worked on together with FourSquare and I have to say it's been really enjoyable working with them.
If you are interested in hearing some of the previous episodes just check out the links below!
From Pixels To Patterns AI In Spatial Analysis
https://mapscaping.com/podcast/from-pixels-to-patterns-ai-in-spatial-analysis/
Big Data In The Browser
https://mapscaping.com/podcast/big-data-in-the-browser/
Spatial Knowledge Graphs
https://mapscaping.com/podcast/spatial-knowledge-graphs/
Designing For Location Privacy
https://mapscaping.com/podcast/designing-for-location-privacy/
All Of The Places In The World
https://mapscaping.com/podcast/all-of-the-places-in-the-world/
Geospatial Jobs
There are a few new jobs on our Job Board! The most interesting one is the role of Social Media Manager at Felt - United States (Remote)
( If you want to apply for this one, it might be a good idea to listen to this episode first ;)
https://mapscaping.com/podcast/felt-upload-anything/ )
See more at https://mapscaping.com/jobs/
As a bonus for reading all the way to the end :)
If you are looking for free terrain data for anywhere in the world you might find this useful
https://github.com/openterrain/openterrain/wiki/Terrain-Data
This is an interview with a senior data scientist from Hub Ocean, a platform that aims to unlock and unite ocean data.
Hub Ocean - as the name suggests is a hub for ocean data
Now we have talked about these kinds of data hubs before on the podcast - Sentinal Hub - a data hub for earth observation data, Microsoft Planetary Computer, Google Earth Engine, Open Topography is data but for topography data …. The concept is not new but also not easy to implement and if they work, these types of data hubs have a gravity to them that becomes more powerful over time.
One of the guiding concepts behind these data hubs seems to be the idea of FAIR data - Findable, Accessible, Interoperable, and Reuseable data
…. But its not enough to ensure that the data is fair I think we should also consider how we can make the results of our research Findable, Accessible, Interoperable, and Reuseable data
If you are not already familiar with Cloud Optimised Geospatial formats it is worth checking out these two episodes.
https://mapscaping.com/podcast/cloud-optimized-point-clouds/
https://mapscaping.com/podcast/cloud-native-geospatial/
How do we get data from a satellite down to Earth? How do we task a satellite?
Today the answer is likely to be via radios and a system of downlink sites or ground stations. As the satellites pass overhead or within “line of sight” data can be sent via radio from the satellite to the receiver on the ground.
If you don’t want to wait until the satellite can see the ground station, you can send your data to a geostationary satellite that can always see a ground station and let it send the data back to Earth.
Radios are tried and tested, they have been used for this purpose since the inception of satellite communication and radio waves can pass through Earth's atmosphere without significant loss!
But … the frequency spectrum for radio waves is strictly regulated, which can limit available channels for communication, and the bandwidth of radio frequencies is limited, which can reduce the volume of data transmission.
What about lasers?
You can send more data faster with a laser, you don’t need to worry about interfering with someone else part of the radio spectrum, and ground stations can be much smaller even human-portable!
But … lasers struggle with clouds and the technology is still relatively new
So what is the best way to communicate with satellites? Radio or Laser? The answer is … it depends ;)
Jordan Wachs, Director of Business Development for SpaceRake.net does a great job adding context to this discussion but perhaps the bigger question here is what will we do when satellites become internet devices, part of the Internet of Things?
What if they were always on always connected in the same way your phone is always on, always connected? What will this enable?
This episode was sponsored by Sponsored by Sinergise, as part of Copernicus Data Space Ecosystem knowledge sharing
People who liked this episode also liked …
How to keep your satellite pointing at earth
https://mapscaping.com/podcast/how-to-keep-your-satellite-pointing-at-earth/
Hyperspectral v’s Multispectral
https://mapscaping.com/podcast/hyperspectral-vs-multispectral/
Sentinel Hub
https://mapscaping.com/podcast/sentinel-hub/
Swing by our website sometime https://mapscaping.com/
There is a general understanding that it is becoming increasingly difficult to extract meaning from all the data we are collecting without using AI.
But what is AI, and how did we end up in a situation where it is identifying wolves from dogs based on the presence of snow in the background of images?
What does this mean for spatial analysis using tabular data?
What is explainability?
This is not a "how-to" do spatial analysis using an AI episode, it is an overview of AI in spatial analysis episode with Vin Sharma, VP of Engineering at FourSquare
https://www.linkedin.com/in/ciphr/
https://foursquare.com/
So why would anyone want to put alot of data into a browser? Well, for a lot of the same reasons that edge computing and distributed computing have become so popular.
You get the data a lot closer to the user and you don’t have to pay for the compute ;)
… this sounds great but as I found out during this conversation it's not as easy as it might seem!
There are a lot of trade-offs that need to be evaluated when moving data and analytics to the client.
Nick Rabinowitz Senior Staff Software Engineer at Foursquare has a ton of experience with this so he volunteered his time to help us understand more about it.
https://location.foursquare.com/
https://studio.foursquare.com/home
If you are not familiar with the Arrow data format it might be worth checking out
Apache Arrow defines a language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware like CPUs and GPUs. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead
Related podcast episodes that you might find interesting include
H3 grid system
https://mapscaping.com/podcast/h3-geospatial-indexing-system/
The H3 geospatial indexing system is a discrete global grid system consisting of a multi-precision hexagonal tiling of the sphere with hierarchical indexes. H3 is a really interesting approach to tiling data that was developed by UBER and has been open-sourced.
Hex Tiles
https://mapscaping.com/podcast/hex-tiles/
If you have not heard of the H3 grid system before listen to that episode first before listening to this one it will add a lot of useful context!
Spatial Knowledge Graphs
https://mapscaping.com/podcast/spatial-knowledge-graphs/
Foursquare is moving away from spatial joins and focusing on building a knowledge graph. If you are not familiar with graphs this might be a good place to start, also its interesting to hear the reasons for the move from spatial joins to another data structure.
Distribution Geospatial Data
https://mapscaping.com/podcast/distributing-geospatial-data/
This is interesting if you want to understand more about distributed databases and some of the strategies for doing this. It sounds complicated but this episode is a really good introduction!
Cloud Native Geospatial
https://mapscaping.com/podcast/cloud-native-geospatial/
This episode give a solid overview of what cloud-native means and some of the current geospatial cloud native formats out there today
I am constantly thinking about how I can make this podcast better for you so if you have any ideas or suggestions please let me know!
Also, I am thinking of recording a behind-the-scenes episode, is that something you might be interested in? if so what questions do you have?
Sounds like a great idea right?
In this episode, Paul Ramsey explains why you shouldn't ... unless you want to ... and how you can ... if you have to.
You can find Paul's blog here: http://blog.cleverelephant.ca/about
Previous episodes with Paul
Spatial SQL
https://mapscaping.com/podcast/spatial-sql-gis-without-the-gis/
GDAL
https://mapscaping.com/podcast/gdal-geospatial-data-abstraction-library/
Dynamic Vector Tiles
https://mapscaping.com/podcast/dynamic-vector-tiles-straight-from-the-database/
Blog posts by Paul about Rasters in the Database
https://www.crunchydata.com/blog/postgres-raster-query-basics
https://www.crunchydata.com/blog/waiting-for-postgis-3.2-secure-cloud-raster-access
Check Out Our Geospatial Job Board!
https://mapscaping.com/jobs/
I am sure you have heard of ChatGPT by now so the hope of this episode is to give you some more context about what is it built on and how it works.
To do that I invited Daniel Whitneck back on the podcast
You can connect with Daniel here
https://datadan.io/
and listen to his previous episode here:
https://mapscaping.com/podcast/an-introduction-to-artificial-intelligence/
This is perhaps the quote for the episode that I have spent the most time thinking about
"We always thought AI would be logical and lack creativity - but it is almost the exact opposite"
This reframes the idea of being wrong to being creative which I think you could argue really depends on the context!
If you have not already played around with ChatGPT it's well worth spending the time to experiment with it ... while its still free ;)
https://chat.openai.com/auth/login
Further listening
If you have not already listened to this episode about computer vision and GeoAI you might find it interesting. Listen out for the discussion around plausible / realistic data and real measurements - I think this gives more context to the use cases for generative AI
https://mapscaping.com/podcast/computer-vision-and-geoai/
You might also enjoy this episode about fake satellite imagery
https://mapscaping.com/podcast/fake-satellite-imagery/
BTW I have started a job board for geospatial people
feel free to check it out!
A knowledge graph is a network of relationships between real work entities and in this episode, you will learn how and why knowledge graphs might be a better choice than spatial joins!
Further listening!
The H3 Indexing System
https://mapscaping.com/podcast/h3-geospatial-indexing-system/
Hex Tiles
https://mapscaping.com/podcast/hex-tiles/
Points of Interest data
https://mapscaping.com/podcast/all-of-the-places-in-the-world/
Dark Data
https://mapscaping.com/podcast/unstructured-data-is-dark-data/
Computer vision is a field of artificial intelligence (AI) that enables computers and systems to derive meaningful information from digital images.
You might think that this is exactly what we are doing in earth observation but there are a few important differences between computer vision and what some people refer to as GeoAI.
This week Jordi inglada is going to help you understand what those differences are and why it's not always possible to use Computer vision techniques in the field of Remote Sensing.
Listen out for these key points during the conversation!
- Why plausible or realistic data is not always a substitute for actual measurements, except when it is ;)
- In computer vision we can learn from the data, in earth observation we know the physics
- To do interesting work in data science you need to - Computer science, applied math, and domain expertise. You don’t need to be an expert in all three but you need to be interested in all three
- Vectors in the machine learning world don’t necessarily have anything to do with points lines and polygons ;)
Sponsored by Sinergise, as part of Copernicus Data Space Ecosystem knowledge sharing. dataspace.copernicus.eu/ http://dataspace.copernicus.eu/
Related Podcast Episodes
Super Resolution
https://mapscaping.com/podcast/super-resolution-smarter-upsampling/
Fake Satellite Imagery
https://mapscaping.com/podcast/fake-satellite-imagery/
Sentinal Hub
https://mapscaping.com/podcast/sentinel-hub/
Google Earth Engine
https://mapscaping.com/podcast/introducing-google-earth-engine/
Microsofts Planetary Computer
https://mapscaping.com/podcast/the-planetary-computer/
BTW MapScaping has started a Job Board!
it's in the early stages but it's live
Jobs - Mapscaping.com
Data is what data does - more about that later on ;)
This episode focuses on designing for privacy, how do we create value from location data without sacrificing personal privacy?
Well, you might start by adhering to the Enhanced Standards For Precise Location Information which means that information about sensitive places like churches, hospitals, military bases, and LGBTQ+ spaces isn't misused or sold. Plus, they protect our exact location from being shared with law enforcement or bounty hunters!
Yes, that's right bounty hunters!
You might also think about adding noise to the data, maybe you want to blur the time stamp and look at everything in aggregate?
It turns out the location data is not just classified as personal data but is actually classified as sensitive personal data in privacy law so if you are going to work with it you really need to understand the risks involved.
One of the really interesting ideas mentioned by Elizabeth Hein VP of Compliance & Data Protection was the idea of regulating Use, Harm, and Risk instead of sensitive data
Data Is What Data Does: Regulating Use, Harm, and Risk Instead of Sensitive Data
If you want to learn more about POI data and why points of interest data are so hard
check out the episode called All of the Places in the World: https://mapscaping.com/podcast/all-of-the-places-in-the-world/
On a side note, I am working on a side project, it's a job board for geospatial people, and you can find it here https://mapscaping.com/jobs/ it's still in the development phase but feel free to check it out!
When comparing multispectral and hyperspectral data it is not simply a case of “more data more better”!
With hyperspectral you have “The curse of Dimensionality” but you also get more flexibility to pick exactly what bands you want to use!
With multispectral you have less noise but you also have less data!
This episode is designed to be a beginner's guide to the differences between hyperspectral and multispectral satellite data. Sponsored by Sinergise, as part of Copernicus Data Space Ecosystem knowledge sharing. dataspace.copernicus.eu/ http://dataspace.copernicus.eu/ You can reach out to Gordon Logie here: https://sparkgeo.com/blog/team/gordon/
Here are some courses that focused on hyperspectral and offer further training
https://eo-college.org/courses/beyond-the-visible/ https://eo-college.org/courses/beyond-the-visible-imaging-spectroscopy-for-agricultural-applications/ https://www.enmap.org/events_education/hyperedu/
Protomaps is a serverless system for planet-scale maps, it's an umbrella project consisting of a few different components one of which is PMtiles.
PMtiles is “Cloud Optimise Geotiff” for web mapping, what this means is that you can build a base map and host it without the need for a server!
PMtiles is a single file that you can access via HTTP range requests in the same way that you can access data within a Cloud Optimised Geotiff with the important difference that PMtiles can also contain vector data!
What this means is that you can create your own base map, and host it on something like Amazon S3 object storage at a fraction of the cost of other base map solutions!
During this episode, you will hear Brandon, the founder, and creator of Protomaps, talk about scarcity, and well I have never really thought about base maps as being a scarce resource I can definitely see how a product like PMtiles could remove some of the barriers to entry for a lot of creativity in terms of base maps.
More information on Protomaps is here: https://protomaps.com/
Tippecanoe
https://github.com/felt/tippecanoe.git
https://bertt.wordpress.com/2023/01/06/creating-vector-pmtiles-with-tippecanoe/
Relevant podcast episodes
Cloud Optimized Point Clouds
https://mapscaping.com/podcast/cloud-optimized-point-clouds/
Cloud Native Geospatial
https://mapscaping.com/podcast/cloud-native-geospatial/
Microsoft’s Planetary computer
https://mapscaping.com/podcast/the-planetary-computer/
Stamen Design - Full Stack Cartography
https://mapscaping.com/podcast/full-stack-cartography/
If you have any questions or comments, let me know, I would love to hear from you!
Personally, I don't feel like aerial imagery gets the attention it deserves! So I invited Michael Bewley - Senior Director of AI Systems at Nearmap back on the podcast to help bring us up to speed on the state of the art of capturing, processing, and building a business around aerial imagery.
If you don’t care about aerial imagery, think of this as a story about turning unstructured data into structured data into insights and building a business around that.
You can connect with Micheal on Twitter and LinkedIn
Listen out for the following highlights
- It's not a camera it's an imaging system!
- Detecting change is not hard, detecting meaningful change is hard
- Are human abilities still a good benchmark for AI systems?
- How to determine if an AI system is a prototype or production ready
Previous Interview with Michael Bewley
Stratospheric Balloons As Remote Sensing Platforms
This episode covers a wide range of topics from the role of geospatial in systems thinking - representing natural systems in location systems and how we can apply the technology behind virtual worlds to the real world.
During this conversation, it became clear to me that technology should be used in service of culture and not the other way around. I think in terms of geospatial we have an interesting opportunity to have an outsized impact if we can just figure out how to market it!
I often think about problems as being either technological or cultural and it seems to me that technology generally moves faster than culture. Although I am sure we can all think of some great examples of huge cultural shifts that have happened in response to short-term emergencies in the past few years.
Culture seems to react to technology as opposed to driving technology. I think if we were all better at marketing we could change this.
Connect with Ingo Simonis
https://www.linkedin.com/in/ingosimonis/
Previous Episodes featuring the Open Geospatial Consortium
The Open Geospatial Consortium
https://mapscaping.com/podcast/the-open-geospatial-consortium/
Open Geospatial standards - shared standards to solve shared problems
https://mapscaping.com/podcast/open-geospatial-standards-shared-standards-to-solve-shared-problems/
How Does Your Phone Know Its Location?Today’s show explains how your mobile device determines your location, commonly displayed on a map using the popular ‘blue dot’. Our guest is Ed Parsons, Google's Geospatial Technologist. He has been at Google for over 15 years; but before that he came from academia, and even helped to set up one of the first GIS courses taught at Kingston University. Prior to that, he worked for the Ordnance Survey and the National Mapping Agency in the UK.
The Blue Dot and GPSThe blue dot on a map shows your location as determined by your mobile device. With the blue dot, you no longer have to manually figure out where you are, as you would have to if you were using a paper map.
The very first mobile devices with the capability to determine location were using GPS (Global Positioning System). To this day, many people still think that GPS is the only satellite constellation that all mobile devices use to determine location. GPS is actually only one of the several GNSS (Global Navigation Satellite System) constellations used to determine location. GPS is an American system; other GNSS include GLONASS of Russia, Galileo of the European Union, and Baidu of China, among others.
Why Does Your Phone Need To Know Its Location?Despite helping you in navigation, your phone also needs to know its location for its basic functions. Knowing a phone’s location, service providers are able to route its calls through the closest cell towers. Messaging and calling is not possible if a phone’s location is not known. Therefore, all mobile networks have to maintain a rough location of where a device is in relation to their cell towers. This is a key way that a device is able to know its location.
Setbacks in GNSSWhen a device is using GNSS to determine its location, it needs a line of sight to the satellite sending the timing signals so as to calculate an accurate location. If there's an obstruction between the device and the satellite, (such as a tree, a building, or a mountain) the device may not get a signal at all, or may suffer multipath problems, which is when the signal bounces off the objects during its journey, resulting in inaccurate timing.
For a device to use GNSS, its radio receivers have to be turned on so as to receive the satellite signals. The power needed to keep these radios on for prolonged periods, as well as the power needed to do the actual computation of location from the satellite signals, can drain the phone’s battery rather quickly. Power management is a major concern in using GNSS, and several alternatives such as using Wi-Fi hotspots to determine location have been developed over the years to try and minimize power consumption.
Location Determination with Wi-Fi HotspotsDetermining location based on Wi-Fi hotspot proximity is widely used today due to the popularity and presence of Wi-Fi. From a home broadband to public Wi-Fi in railway stations, airports, or coffee shops – there are Wi-Fi hotspots all over. Each Wi Fi hotspot has a unique identifier – an address that is part of the internet’s network infrastructure. The Wi-Fi hotspots and their locations can be built into a database, and cached into a mobile device for use in location determination. There are a number of providers of the databases that match Wi -Fi hotspots to locations. A device just needs to keep its database updated to capture the changes when Wi-Fi hotspots are moved around.
Using Wi-Fi hotspot to determine location is more efficient in terms of power consumption than using GNSS technology. In most cases, people already have their Wi-Fi on as that is how most people access the internet today. This means the device does not use a large deal of extra battery power to find its location. Since the phone switches between the location determination technologies in the background, where you may be thinking that your device is giving you a location coming from GNSS technology, it is most likely the Wi-Fi infrastructure and technology identifying where you are.
Error CorrectionSometimes a device displays an incorrect location before correcting itself in the background and updating to the correct location. A common cause for this sudden shift is when your device initially picks up a stale Wi-Fi location, especially on startup. By checking against the nearest cell tower, it disregards the location from the stale Wi-Fi and updates to your correct location.
For multipath problems when using GNSS, using a model of a city’s morphology can help in error correction. The model gives insight on how the satellite signals bounce off objects in the area. The multipath errors caused by these bounces can then be canceled out and corrected for, increasing the location accuracy.
Fused Location ServiceCell towers, Wi-Fi, and GNSS constellations are the three main technologies used to determine a device’s location. A fused location service means that the choice of the technology being used at any particular point in time is largely invisible to the user, and in many cases from the application developer as well.
The device’s choice of which technology to use is a capability at the operating system level. All that is needed at the application level is making a single call to the operating system with a request for the location level that is required. Depending on whether the application needs a precise location or a relative location, the operating system will decide on the best technology for providing the location at the required accuracy. Hence, it is very likely that a device will only use GNSS to get location when precise location is needed, for instance, if an application is giving turn by turn directions. If proximal location is needed, Wi-Fi will most likely be used due to its power efficiency.
Visual PositioningVisual positioning uses the phone’s camera to sense the environment around it. It makes a comparison between what the camera sees against a database containing images of that street. The device can then orient itself to the correct position and orientation during navigation. Google in particular has a rich resource of street view imagery of many cities and locations around the world. Google Maps users can use visual positioning to orient themselves correctly and know which side of the street they are on, and which direction they are facing. The visual positioning system supplements other location services to not only identify your location, but also the orientation (i.e. the compass direction to which you are facing or heading).
Do You Always Need A Precise Location All the Time?Precise location is not always what we need for most of our day to day location needs. Now, if you are doing turn by turn navigation as a pedestrian, then you definitely need precise location. If your goal is to get a local weather forecast, or find your local Starbucks, then a proximity location that puts you on the right street would suffice.
COPC – A Cloud Efficient Data FormatIn this episode, the discussion revolves around cloud-optimized point clouds. Our guest is Martin Dobias, the CTO at Lutra Consulting. Coming from a background in computer science, and a passion for geospatial, Martin has been part of a team that has done a ton of interesting work in the open source geospatial world. Today, he shares about their latest, state of the art developments in working with point clouds on the web; Cloud Optimized Point Clouds (COPC).
What Does Cloud Optimized Point Clouds Even Mean?Point clouds are sets of individual points plotted in 3D space. They are typically very large datasets, as they must capture a real space in great detail. For instance, point clouds for a whole country may easily be several trillions of points, and many terabytes of data. Handling these large datasets on the web requires a lot of bandwidth to download and process.
The files for Cloud Optimized Point Clouds are structured with indexes for each part of the dataset. The index structure makes it possible to stream only the parts of the data that are required, without having to download the entire dataset.
What is Point Cloud Indexing?Point cloud indexing structures a point cloud file, making it possible to find any particular point of interest in the file without having to scan through the entire dataset. COPC files are internally indexed using a 3D structure of cubes called Oak trees. One part of the file is the data itself, and the other part contains the hierarchical information of where to find each cube.
At the root level of the hierarchy is a single cube, which at the next level is split into eight smaller cubes. The splitting continues subsequently up to the highest hierarchical level. As the cubes get smaller, they contain a smaller amount of data, which saves bandwidth if a user is only interested in a small part of the data. It is similar to traditional tiling, but with the added 3D context.
How COPC Files Are Accessed Using HTTP Range RequestsRange requesting is a feature of the HTTP protocol used to access information more efficiently from web servers. Instead of a server sending an entire file to a browser, the range request feature allows the browser (client) to define a specific part of the data that the user is interested in. Subsequently, only the requested part is sent by the server.
For cloud-optimized point clouds, the server will simply go through the hierarchy and find the cube or multiple cubes that satisfy the request, and return these. This process is much faster than having to search through the entire dataset. Moreover, only the point cloud cubes that satisfy the browser request are sent by the server, which reduces the bandwidth used.
Converting LAS Files to COPCLAS is a standard open source format for point cloud data interchange. However, it is less efficient to work with on the cloud since the data in it is not indexed. This means the whole LAS file must be loaded before it can be queried, sidestepping the efficiency we see with COPC indexing.
Conversion of LAS to COPC can be done in QGIS using Entwine. Practically, when a point cloud file is loaded to QGIS, it is automatically converted to COPC. QGIS structures the data automatically in order to make operations more efficient as opposed to working with unorganized datasets.
When point cloud files are converted to COPC in QGIS, the new COPC file contains all the information in the original dataset. Unlike other software that may discard some information when processing a file, for COPC nothing is discarded. This makes the COPC format great not only for visualization purposes, but for analysis as well.
What Infrastructure is Required to Serve COPC Data?Serving cloud-optimized point clouds does not require any special infrastructure between the server and the client. It is easy to host and get the data to the client without complex infrastructure, i.e. there is no need for something like GeoServer, MapServer, or QGIS. Just having the COPC data in blob storage somewhere is all the infrastructure that may be needed.
Compatibility of COPC FormatA compressed LAS file is called a LAZ file. COPC is much like a LAZ file. This means that applications that accept these formats will also be able to work with COPC files without having to implement special support. The only difference is that they will not be able to use the extra features of internal indexing in the COPC file.
Where Can You View COPC data?
QGIS offers support for viewing COPC – both stored locally on your device, or remotely in the cloud. Using a link that points to a server containing COPC data, QGIS will load the data on demand according to the queried range. The data is further cached in QGIS, which makes subsequent data loads and views much faster.
There are also a couple of projects coming to life that explicitly support cloud-optimized point clouds. An example is the web viewer built by Hobu. With the link to a COPC data server, the web viewer will fetch the relevant COPC files and render them in the browser.
The PDAL LibraryPDAL (Point Data Abstraction Library) is a library that contains a set of tools for working with point cloud data. In the QGIS environment, many users are familiar with PDAL’s feature for the data access of point clouds. Many of the library’s other functionalities are unknown to a lot of users. It contains a dozen features to classify, filter, export, convert point cloud to raster or meshes, amongst others.
The main reason why many functionalities in PDAL are not popular among ordinary users is due to the complexity in using it. The PDAL library uses pipelines that need to be crafted manually when working with point cloud data. While this may work well for advanced users, ordinary users find it a bit too complicated.
After a successful crowdfunding campaign, Lutra Consulting and several other partners are working to reduce this complexity, and make the functionalities in PDAL more user friendly. The project seeks to build a simple integrated toolbox within QGIS for point cloud data processing. The same way there are integrated toolboxes in QGIS for working with vector or raster data, there will also be one for working with point cloud data. These developments may be expected across the next two QGIS releases; in February and June of 2023.
What is the STAC Protocol?STAC (SpatioTemporal Asset Catalog) is a protocol for easy access to spatial and temporal data. It makes it easier to index, discover, and work with geospatial information. STAC is commonly used with satellite imagery but recently it is increasingly being used for distribution of point clouds as well.
How Were Point Clouds Streamed Before COPC?No doubt, before COPC there were some existing formats for streaming point cloud data. In the open source world, one of them is the EPT format, built by Hobu Inc.. The EPT format closely compares to raster tiles; but for point clouds. It is structured in a big directory with individual files (tiles). COPC files have an advantage over EPT format, as opposed to having thousands or even millions of files in a folder structure, COPC is just a single file – which is much easier to work with. In the proprietary world there are a couple of formats as well, one being the I3S format from ESRI, which supports point cloud data and other 3D data. There is no doubt we will continue to see explosive growth supporting point cloud data management. Stay tuned with the MapScaping Podcast to make sure you stay current on the latest and greatest developments!
PDAL - Point Data Abstraction Library
Full Stack Cartography – Think Like a Designer for Better MapsThis episode features Alan McConchie, a leading cartographer at Stamen. The discussion centers on mapping and visualization; and explores the usefulness of adopting a design-first thinking approach to the map making process. Alan has over 8 years experience making maps that are as beautiful as they are useful. Together, we will gain a deeper insight into full stack cartography, and how we can leverage it in our own work.
Spatial Data Visualization vs. Non-Spatial Data VisualizationAlthough they are both ultimately design tasks, there are some distinct differences between spatial and non-spatial data visualization. For one, there is lesser liberty in spatial visualizations than there is in non-spatial data visualization. Unlike in the latter where the designer can take liberty in choosing where to place objects and what shapes to use, spatial visualizations usually have to maintain location integrity. If the locations in a map are distorted, the map will be misleading and fail to serve its purpose. The flexibility in maps is mainly found in the choice of the symbology, color ramps, and fonts used. Even these areas are often guided by standard cartography rules, so it's important to make the most of the creative space provided if you want your maps to stand out from the rest.
What is Full Stack Cartography?Full stack cartography is seeing the world as a challenge of design, and how it can be effectively mapped for a particular use case, or to communicate your topic of interest. To back it up a bit, full stack is the idea of taking your entire workflow, and all of the areas of your system that it touches into consideration together. Full stack cartography incorporates design thinking into the map making process right from the beginning, throughout the process, and until the end. Writing the code and scripts used to process the map data is heavily guided by the use case of the end product.
Ideally, every decision, at each stage of the process should contribute towards making the end product the best it can be for its use case. All the decisions made when making a map are important; from how the data is stored in a database, to how it is processed, to how the end product is displayed. The ripple effect of all these decisions throughout the process can make achieving the end goal easier (if done right), where ignoring them can make the process more difficult, inefficient, and altogether frustrating.
Why Are Base Maps Pre-Generated? A base map is a pre-generated background map that provides context to other map layers that are overlaid on top of it. They generally contain generalized land uses, streets, water features, and building footprints, amongst other landmarks. Pre-generating base maps makes loading web maps faster, and more efficient, by leveraging imagery tiles. Since creating base maps is a complicated process that usually requires a huge amount of data, generating them on the go every time is computationally intensive and slow for devices with low to standard computing power – which is the case for the majority of users.
In order to improve the user experience even further, base maps are divided into many small pieces, called map tiles, so that a browser does not have to load the entire base map, but rather only loads the pieces (tiles) of the area that a viewer is interested in. This is faster and more efficient than loading the full dataset.
How Do Map Tiles Work?Using map tiles is a popular and common technique for displaying maps on the web. After generating a map, it is divided into many small pieces (usually square shaped, be sure to look into Hex Tiles); which, when joined, form a seamless single map.
Tiling maps improves the user experience by reducing the size of the dataset that a browser has to load when displaying a map. Less data to load means displaying a map is faster, and uses less bandwidth. This is significant as web maps are already data and display intensive due to the many additional features and layers that need to be loaded alongside the base map, especially if those layers contain a lot of attribute data. To view an area of a map, the browser will only load the tiles that cover that area. When a user pans the map, more tiles that cover that area are returned. Previously loaded tiles may be cached so returning to your previous extent will be faster than if the map was loaded fresh.
How Are Map Tiles Created?After creating a map, tiles can be generated using map tiling tools or services such as Mapbox Tiling Service. Tiles can be in a vector or raster format. The tiles are generated for different zoom levels to allow users to zoom in or out of a map as they may need. At higher zoom levels, the map tiles are further split into smaller equal tiles that display more details. Each zoom level has its pre-generated tiles that are easily loaded to a browser upon request. Most programs, like ArcGIS, have a default tiling scheme and thus create tiles for specific zoom levels that are most optimized for their viewers. If, however, you want your map to be drawn at different intervals, you can create and publish a custom tiling scheme.
Geometric SimplificationCreating map tiles includes deciding which details will be included in the tiles, and which will be filtered out. In order to improve efficiency even further, cartographers try to reduce the size of data stored in each tile as much as possible. Some simplification is applied in the lower zoom levels to exclude details that may not be visible or useful in that zoom level. For instance, an application to show bike trails in a park may only display the trail lines at lower zoom levels, but then include the attributes like the trail names, or topographic lines at higher zoom levels. A cartographer decides which details to include at each zoom level and cuts out what is not required to avoid storing unnecessary data in the tiles. Having light weight tiles makes them even more efficient, fast, and useful for many mobile applications.
Styling Map TilesAfter deciding what map details appear at each zoom level, it’s time to decide how they should look. The tiles only contain the shapes and attributes of the map features. In order to apply styles such as colors and textures, a stylesheet is used. The stylesheet defines how the data stored in the tiles will be displayed. Design tools such as Mapbox Studio, or the open source option Maputnik, can change the color, line thickness, and other styling characteristics of a feature and display them instantaneously for review.
What is a “Good Map” in Cartography?A good map may mean different things in different use cases, and to different people. Generally, a good map is one that finds the right balance between beauty and usefulness. No doubt we all love a beautiful map, but if it is too beautiful then users spend more time admiring it than actually using it. This is not a good map as it does not successfully and efficiently convey its message. Keeping the distractions to a minimum is one important trait of a good map.
Design-first thinking helps to create good maps that are useful, as well as visually appealing. Everything from the choice of colors, to fonts, to avoiding emphasis on less important features will contribute to whether a map is successful in its application. The ultimate measure of a good map is how well the map serves its intent. This of course includes making it visually appealing so that people are willing and able to use it in the first place.
Is Cartography Changing?In the world of cartography, a lot has changed already. The tools used to make maps decades ago are not the ones being used today. The way maps are displayed today is different from how they were displayed in the past as well. The way maps are used have evolved too, and this is why the approach to cartography has also evolved over the years.
Today, maps are not only created for humans but for machines as well. For instance, self-driving cars need maps for navigation, and have to process a lot of simultaneous inputs.
In the future, more changes can be expected as further advancements are realized. Technology is getting better by day and soon we may see even more new products in cartography. For one, since personalization is a big thing these days, we may start seeing more personalized maps. This will come hand in hand with the loss of privacy that seems commonplace, and even expected nowadays. The developments seen in indoor mapping suggest we will soon see seamless outdoor-to-indoor navigation.
There is much we can expect from the future. In order to avoid being surprised, get involved now.
Recommended Podcast Episodes about Cartography
Applications of thermal imaging from space include monitoring wildfires, urban heat islands, economic activity, and the built environment.
But it's not easy ;)
Connect with Robin Cole at
https://robmarkcole.com/
Check out the Earth Observation Hub!
https://geoawesomeness.com/eo-hub/
More Geospatial Podcasts episodes
Recommended Podcast Earth Observation Podcasts
Fake Satellite Imagery
https://mapscaping.com/podcast/fake-satellite-imagery/
The LandSat Program
https://mapscaping.com/podcast/the-landsat-program/
How To Keep Your Satellite Pointing At Earth
https://mapscaping.com/podcast/the-landsat-program/
The problem of unification: Spatial data comes in many different sizes, shapes, and formats making it a difficult and time-consuming process to join data for visualization, exploration, and analysis.
Enter the Hex Tile system!
Contact Foursquare: connect.foursquare.com/mapscaping
Check out Unfolded https://foursquare.com/products/unfolded/ https://www.unfolded.ai/
Introducing Hex Tiles: https://foursquare.com/article/introducing-hex-tiles-our-next-gen-tiling-system/
Example Maps
https://studio.unfolded.ai/public/e86787a1-4871-4fbb-8eb8-33445e81ca73
https://studio.unfolded.ai/public/e3a6d2df-1784-4686-972b-9d3793c2515d
https://studio.unfolded.ai/public/beb3cf0d-211d-4e5c-b08d-663da9a0631d
Recommended Podcast Episodes
Dynamic Vector Tiles Straight From The Database
How to Keep Your Satellite Pointing at Earth
The guest on this show is Jack Reed, a PhD student at the MIT Media Lab. He started out in mechanical engineering and then later on moved into aerospace engineering. At the MIT Media Lab, he is part of an interdisciplinary research group alongside lab mates with backgrounds in data science, ethics, and art. Together they work on making space sustainable, and using space-based assets and imagery to help promote sustainability on Earth.
Why do We Need to Orient Satellites?
We tend to assume that since space is a vacuum, then there is nothing that would make the satellite drift from its original course. If this is the case, then there should really be no need to worry about orientation since the satellite should keep pointing to the same direction.
In reality, satellites are affected by forces like atmospheric drag, especially in low Earth orbits. Atmospheric drag slows down satellites and pulls them out of orbit.
Another reason to orient satellites is that as they revolve around the Earth, at 180 degrees they would be pointing directly away from the Earth. Unless they are rotated it will have to go another 180 degrees before they point directly at earth again.
In order to keep the satellite pointing at Earth at all times, they need to be rotated constantly, otherwise, they lose half of their utility.
How Satellites Orientate Themselves in Space?
The ability to control a satellite’s position in three dimensions, and where it is pointing is critical to increasing its lifespan. Controlling it in three-dimensional space helps to keep the satellite in the correct orbit, while controlling where it is pointing to ensure that the satellite is capturing the right data, and sends communications back in the right direction.
If it goes off its orbit, or points in the wrong direction and is unable to get back, its lifespan would be cut short, and a huge amount of resources would be wasted.
Propulsion is used to move a satellite through three dimensions. It enables satellites to get back to orbit if they get off. Propulsion can be achieved by having rockets or thrusters at different corners of a spacecraft for turning. Large spacecraft also use propulsion to control where they are pointing. Ideally, engineers want to achieve propulsion with the least possible amount of fuel, as the weight of the fuel at launch can make the endeavour far more expensive and complicated.
When talking about flight, attitude is the information about an object’s orientation and position on its axis in relation to the plane below it. These properties are pitch, roll, and yaw.For attitude control, most satellites use reaction wheels. Speeding up or slowing down a reaction wheel on a spacecraft, will cause the spacecraft to rotate in the opposite direction. By having at least three wheels, a satellite can precisely orient itself towards wherever it needs to be. You could hypothetically call this an attitude adjustment.
How Do Satellites Determine Their Attitude?
In order to keep pointing at Earth, satellites need to know how they are oriented relative to the Earth’s location at any particular moment. A spacecraft subsystem, the ADCS (Attitude Determination and Control System) serves the purpose of maintaining a satellite's optimal orientation. It consists of IMUs (Inertial Measurement Units) and a variety of other sensors working together to tell where a satellite is pointing in relation to the Earth’s location.
Some of these sensors include:
Horizon Trackers
Since the Earth is warmer than space, infrared cameras can see the cut off of the earth’s horizon in space. In this way, a satellite is able to figure out where the Earth is. The downside though is that due to the horizon’s huge size, it does not give a precise point as to where all satellites will be pointing, producing a more generalized target direction.
Sun Trackers
The sun can be tracked through sensors that look for the hottest thing in the sky. Unlike the Earth’s horizon, which may appear as a huge circle of horizon, the sun tends to be in a very particular spot in the sky that the satellite can point at. By tracking the sun, a satellite can determine how they are oriented in space. During the times in its orbit when the Earth is between the sun and the satellite, sun tracking cannot be used.
Star Trackers
Star tracking is the most accurate method that satellites use to determine their attitude and figure out the location of the Earth. In fact, even the early astronauts using sextants, an instrument used for measuring the positions of stars since the 18th century. Star trackers compare the position of very particular stars against a star catalogue in order to tell the direction in which a satellite is pointing.
Magnetometers
Satellites on Earth orbits can use the Earth’s magnetic field to orient themselves. A magnetometer which detects the Earth’s magnetic field can tell a satellite which direction is north, and with this, a satellite can determine its orientation.
Earth based Ground Stations
Broadcasted signals from ground stations on Earth can be used by satellites to figure out where the Earth is relative to their current position. A major limitation to this method is that the signals’ strength fades out the further the satellite travels, and many ground stations are needed to keep continuous coverage and not lose data.
Can GNSS Satellites Be Used to Orientate Low Orbit Satellites?
GNSS constellations live about 20,000 km above Earth, and hypothetically they can potentially help orient low orbit satellites, which are usually below 2000 Km. While it is possible, there are certain caveats that limit using GNSS satellites from orienting lower satellites. Firstly, GNSS satellites are broadcasting specifically to Earth, so it is possible to receive a signal from at least four of them at any point on Earth, and triangulate an accurate position. This is how we use GNSS for earth navigation systems, and GPS. As you move higher, the field of view gets narrower and satellites which are very high up may not receive signals from enough GNSS satellites to triangulate an accurate position. Additionally, GNSS predominantly gets a satellite’s position, but not its attitude or orientation.
Want to learn more about using GNSS on earth? Listen here to learn about Where Does the Blue Dot Come From?
Edge Computing
Tones of high-resolution imagery is being captured by earth observation satellites every day, and edge computing has an important role to play here. It is not necessarily feasible to transmit everything that a satellite captures, since some of the data may not be usable. For instance, running algorithms on captured images out in space will help to sort out the ones that have too much cloud in them, and only downlink the images that would be usable. Algorithms can also complete some light processing and correction of the images, so the data is already in a more usable state before being sent down to a ground station on earth.
Space and Geospatial
Compared to previous generations, we are certainly living in a golden age of both geospatial data, and the space industry. There are a lot of commercial players that are designing, building, and launching new Earth observation satellites; and many others who are figuring out new and innovative ways of processing that data and turning it into useful products for various applications. Earth observation satellites are surely becoming an important pillar for many geospatial applications.
Want to learn more about the growth of earth observation, remote sensing, and imagery? Listen to our podcast with Dr. Aliastair Graham
Recommended Podcast episodes
This is a story about bathymetric Lidar... and how geo-tagged sharks led to the discovery of a huge nature-based carbon sink in the Bahamas.
More on this case study here: https://r-evolution.com/r-initiatives/oceans
The Ocean of Things
https://www.darpa.mil/program/ocean-of-things
Recommended Podcast episodes
Mapping The Ocean Floor
Mapping Oceans With Sound And Mapping The Sound In The Oceans
PDAL -Point Data Abstraction Library
Whitebox Tools Is The Backend To Many Frontends
consider supporting this podcast on Patreon
https://www.patreon.com/MapScaping?
Or go to MapScaping.com to find out about sponsoring our website
reach out on Twitter https://twitter.com/MapScaping
or LinkedIn https://www.linkedin.com/in/danielodonohue/
Sentinel Hub – The Cloud API for Analysis Ready Satellite Data
Gregor Milcinski is the CEO and co-founder of Sentinel Hub and has worked in the geospatial field for about 20 years. He shares his knowledge of the Earth observation industry and gives us an in-depth explanation of what it is that the Sentinel Hub does.
Sentinel Hub is a cloud API for satellite imagery. It uses its APIs to enhance access to satellite data from missions such as Sentinel, Landsat, and other commercial satellite projects. Sentinel Hub users make use of its APIs to process the data they are interested in, and access it in a format that best provides the information they need.
This helps take off the operational load in terms of time and effort required to process the large, ever-changing volumes of satellite data. Sentinel Hub enables users to obtain Analysis Ready Data (ARD).
This means having data in a form that is ready for a particular workflow process and does not require additional cleaning. Sentinel Hub estimates that the capabilities they provide cover about 80% of what users may want to do as far as data processing is concerned.
Among others, these include Ortho-rectification, data transformations, rescaling, re-projection, as well as applying some machine learning models. Using a simple API request, users can get immediate access to full and global archives of the relevant satellite missions partnered in Sentinel Hub.
A Peek into How Sentinel Hub was Built The idea that eventually transformed into Sentinel Hub was born from operational difficulties in a previous project. This triggered the realization that the technology which was already available for working with satellite data was not overly suitable for processing the ever-expanding, continually updating, large volumes of data collected by satellites. A prototype was developed and subsequently iterated to evolve into the now Sentinel Hub.
This journey of growth is an inspiration to those who may want to build something, but are unsure on how they should start. Do not get stuck in the idea that you have to start big straightaway. Start where you are confident, and once you figure out what is working, keep building on it.
How Sentinel Hub Works Sentinel Hub works ‘on the fly’. It does not store pre-processed data that can be ordered off-the-shelf. Rather, the processing happens on the fly as requested by the user. This approach was chosen since it is quite impossible to accurately predict what a users’ needs will be in terms of geographical, temporal, and spectral aspects.
This is also one of the reasons why Sentinel Hub does not cache processed results (apart from privacy policies). Considering these aspects, there is a high risk of storing petabytes of data that may never be needed. Instead, Sentinel Hub worked to optimise their processing steps as much as possible for speed.
Currently, it only takes a couple of seconds for the most usual requests (like above) to process. This is typically fast enough that users can integrate these APIs directly in their applications in an interactive manner, without even having to store the data on their side.
Keeping operations to "on-the-fly" also allows Sentinel Hub to expose their new capabilities and improvements to their users by simply deploying a new version.
If data was pre-processed, there would be a fixed dataset which would need processing in order to introduce the new improvements. This could be expensive, time-consuming, and inconvenient.
How is Sentinel Hub Different? The description of what Sentinel Hub does may have you thinking that it is the same as Google Earth Engine, or Microsoft’s Planetary Computer. Ideally, it actually falls somewhere in between the two. See the sections below to see how they compare.
Sentinel Hub vs Google Earth Engine Google Earth Engine was designed as a search platform where people could do analyses, and then make use of the results. It was not primarily designed to power other applications.
Even though there are workarounds regarding this, Google Earth Engine was mainly designed for people to work with the results of its analyses. On the flip side, Sentinel Hub is designed to power other applications.
It offers a set of APIs which users can easily integrate into their procedures and workflows. It removes limitations on what users can do, and where they can do it. Building your workflow on top of Sentinel Hub, you can do just about everything, since it is possible to fine-tune what you'd like to get from Sentinel Hub APIs.
Sentinel Hub vs Microsoft’s Planetary Computer Microsoft’s Planetary Computer can be viewed as a platform that is mainly for making geospatial data available in a cloud-native way. It does not abstract the complexity of satellite data, which makes it difficult to perform tasks like stitching scenes. Microsoft’s Planetary Computer provides APIs to access the metadata of the satellite data for users’ workflows being run on virtual machines in Azure. This in itself is very limiting for users who may want to run it on a different cloud. On the other hand, Sentinel Hub allows users to be more flexible as they can use the program in their own environments.
Businesses that have their own infrastructures and proprietary data that they do not want to use outside their environments can simply use Sentinel Hub as a data stream in their procedures from wherever they are.
Who Are the Users of Sentinel Hub? The vast majority of Sentinel Hub users are application developers and data scientists. The application developers are mostly those working in the agricultural sector.
One reason for this is that Sentinel-2 provides a powerful collection of data with a resolution that is most useful for agriculture and vegetation monitoring. Typically, many of the Precision Agriculture applications are powered by Sentinel Hub.
The use of Sentinel Hub by data scientists leans towards agriculture primarily for monitoring activities and climate change analyses. Data scientists are the largest consumers of Sentinel Hub since they perform large scale operations using machine learning. Despite making up only 5% of the paid users of Sentinel Hub, data scientists consume about 80% of the volume of data.
Apart from the agricultural sector, Sentinel Hub also serves the energy, defence, environment, and mining sectors.
Partnering with Sentinel Hub Satellite companies can partner with Sentinel Hub through its “Bring Your Own Data” service. This service allows satellite companies to plug in their data to Sentinel Hub in any cloud native format supported by Sentinel Hub (i.e. GeoTIFF, XAR), and basically any kind of raster data. Sentinel Hub does not copy this data, but merely stores the metadata in their database for faster access, like a catalog.
Users can then make use of the same set of features offered by Sentinel Hub to distribute data efficiently. The satellite providers are responsible for charging users for using their data as Sentinel Hub does not support this functionality at the moment.
Sentinel Hub’s EO browser The EO browser is a Google Maps-type application that runs in a browser. It provides access to the satellite missions supported by Sentinel Hub, and is free for non-commercial use. In the EO browser, you can check for the most recent data in any part of the world and visualize it interactively (i.e. zooming in and out, viewing different band combinations, doing time lapses, switching between satellites), and much more. No account is needed to use these capabilities in the browser.
The EO browser showcases what Sentinel Hub’s APIs can do when integrated into workflows. It is a perfect display of how satellite data can be used. If you want to get familiar with or know more about satellite data, the EO browser is a useful resource for that.
An Overview of the Earth Observation (EO) Industry The usefulness of satellite imagery has grown over the years as experts explore more applications where the use of satellite data would improve efficiency, and insights. Coupled with developments in IT and data capture technologies, the Earth Observation industry has exposed new opportunities but also faces some setbacks.
Opportunities in the EO industry Spatial resolution in satellite imagery has improved in recent years and has unlocked new opportunities for using satellite data. Some satellite missions, such as Sentinel-2, have launched a revolution by providing very useful, good quality, free data set that can be used as the basis for a number of geospatial workflows.
Machine learning is also worth mentioning due to its contribution to the EO industry. It has revolutionized the interaction with the normally large volumes of satellite data, and processing it. There is still much to be done in terms of developing machine learning procedures for highly dynamic satellite data, and the wide variety of more niche uses that consumers crave.
Setbacks in the EO Industry The complexity in the nature of satellite data, coupled with the unique traits of geospatial applications introduces difficulties in using machine learning to fully realize opportunities that emerge in the EO industry. This uniqueness hinders the development of off-the-shelf machine learning algorithms. There are a very limited set of applications for which machine learning procedures can be offered as a service (i.e. ship or building detection). In most other cases, there would still be a need to tailor the procedure to a specific workflow or niche.
In an effort to contribute to the development of machine learning in geospatial applications, Sentinel Hub have open-sourced all their internally developed machine learning procedures, and hosted them on GitHub. They allow other people to refine these procedures and use them as a starting point for their own machine learning projects.
Making the decision on how to balance temporal and spatial resolutions during data capture is also a defining line on whether the missions will generate sufficient value.
For instance, applications in the construction industry may require high spatial resolution (i.e. 1 meter) as well as a high temporal resolution (i.e. 1 day) to monitor construction progress. For applications like vegetation monitoring where there will be no significant change within a day, using these exact resolutions will be overkill.
It is highly likely that the value generated may not be sufficient to compensate for the cost used in processing.
Another challenge is the fact that satellite data is only periodically produced. It is unlikely that a user can find what they want in their area of interest at the particular time that they are interested in. A satellite can only be tasked for the future, not the past.
Advancing the Earth Observation Industry Sharing knowledge has always been one of the pillars for new developments that can take any industry to the next level.
In the geospatial field, more and more companies are cultivating the culture of sharing knowledge. Sentinel Hub is already making strides in this by sharing a lot of what they do (i.e. openly sharing their machine learning procedures, which they could easily choose to make proprietary).
Sentinel Hub explained that they are doing this because they believe the Earth Observation field is in a phase where contributing to its growth is more important than growing one’s position within it.
The Future of the Earth Observation Industry Growth is expected in the Earth Observation industry for the foreseeable future. With the introduction of the monitoring approach in almost all elements of our lives, satellite data will be needed since it is a rich, objective, and useful primary source.
The industry will grow since this data will be needed in a form that makes it easier to extract information, and be processed it in a smart way. Sentinel Hub acknowledges that they are only doing a tiny part in the industry. It will need effort from the geospatial community to grow all the tiny parts and realize growth across the entire industry.
Recommended Podcast Episodes
Our guest on the show today is Chris Holmes, the Vice President of Product and Strategy at Planet. Chris entered the geospatial arena almost 20 years ago as an early contributor to the GeoServer project. In the beginning, he was writing code, but eventually discovered his talents were better spent helping to build a community and awareness around GeoServer and the greater Open Geospatial Consortium (OGC). Chasing innovation, he now works to promote wider spread adoption of cloud native geospatial solutions such as the SpatialTemporal Asset Catalog (STAC), and use of cloud optimized geotiffs (COGs) from his position at Planet.
The Basics of Cloud Native Geospatial As one of the newest technologies on the GIS scene, cloud native geospatial can seem a bit intimidating. Realistically, it is a lot of familiar industry staples repackaged to take advantage of huge advancements in computing technology.
At its core, cloud native geospatial (CNG) is your classic geospatial infrastructure, without all those pesky computational power and storage limitations.
By leveraging the power of AWS, BigQuery, Snowflake, Google Earth Engine, or ArcGIS Server, users can access and analyze global-scale datasets without needing to purchase and maintain the physical servers traditionally used in on-premise setups.
This shift reduces the barrier to entry for scientists, and even casual users, allowing more spatial questions to be asked and answered.
CNG systems are flexible and scalable to most needs. One can choose to host some, or all of their data in the cloud, then access it for local analysis, complete that analysis in the cloud, or take a hybrid approach.
GIS work has traditionally followed a desktop-centered workflow. Using cloud native geospatial, it does not matter if you access and analyze your cloud data through a browser, or desktop application, although each platform will come with its own natural limitations. GIS Data Formats and the Cloud Cloud technology alone has existed for a while, but took a bit of a learning curve to adopt into GIS due to the specific requirements and preferences of cloud infrastructure.
This means that some people took this rare opportunity to essentially start from scratch, and build data formats that are optimized specifically for cloud systems, but often still maintain the flexibility to be backwards compatible to desktop and enterprise workflows.
It is not possible to talk about CNG without talking about Cloud Optimized GeoTIFFs (COGs). COGs are the backbone of cloud native geospatial, and are essentially responsible for starting the GIS industry’s race to the cloud. The beauty of COGs is that they can be used as a regular GeoTIFF in a desktop setting, or leveraged in the cloud to unlock fantastic real-time data streaming and analysis.
The key element of COGs is their compatibility with range requests. The efficiency of range requests is what enables streaming in a lot of our favorite applications, like Spotify, and even Netflix or YouTube. A range request is when a client reaches out to a server for information in its HTTP header to first know if the server supports range requests.
If range requests are supported, this means that the client can query what is essentially a table of contents for the data to retrieve only the data that the client is interested in, then stream it back for use.
If you are familiar with Python, you can think of this almost like slicing a list [x:y]. By pulling only the data relevant to the query, performance is greatly increased as fewer packets can be transferred back to the client.
At this time, streaming optimized geospatial formats are mostly limited to raster and point cloud data. There are nevertheless hopes that we will see options for optimized vector formats in the not too distant future. Open vs Closed Geospatial Data Standards In the past, GIS was often more or less siloed within an organization, making it reasonable to work in closed or proprietary formats. Today, in an increasingly interconnected world, sharing data and preparing it with interoperability in mind is essential.
Open data standards have been embraced by many as they can be used as-is, or maybe modified to work with existing infrastructure to create a better fit for an organization’s needs.
The Open Geospatial Consortium in particular has been instrumental in expanding the prevalence of open formats by publishing and documenting open data standards.
Having this groundwork in place gives developers a good spot to start from when creating a custom implementation, and leads to greater potential for innovation.
Although open data standards and formats have come to dominate the industry, closed data standards still have a presence. The best example is Google Earth Engine. Within its system, Google Earth Engine ultimately handles data in a closed proprietary format.
They know, however, that consumers today are generally unwilling to accept the risks of holding all of their data in a closed format.
After all, if the provider went out of business, then the consumer would lose usability of their data. Google remedies this by allowing other data formats to essentially port into their own, allowing clients the flexibility and security they would get if they utilized open standard formats. What's Next for Cloud Native Geospatial? CNG has already brought a huge paradigm shift to the industry, removing traditional access, storage, and processing barriers for those who enter the game. As more and more datasets are uploaded into the cloud, it begs the question of what the next great leap forward will be.
One potential development we may hope to see in the future is data that is optimized for retrieval by search engines. At this point, the data exists, but it needs to be described in a way that allows the search engines to find it, and match it to user needs. This means fully populated metadata, and plain text descriptions that allow data to be matched to a query.
More accessible and queryable geospatial datasets could play a huge role in bringing geospatial to the masses, rather than following the current theme of only being used by those involved in the GIS industry. The SpatioTemporal Asset Catalog is a great example of what this may look like.
Another glimpse into the future of CNG is that eventually, pretty much all the data we could want will be available in the cloud, in open data standards. This can allow the focus to shift from data aggregation, to creative data analysis and applications. As young people come into the industry, they will not be clouded with the ideas of what cannot be done, but will rather see the wealth of options and resources available, and take it and run with it, hopefully leading to the next big thing.
Daniel:
Hey, Chris, welcome to the podcast. Today we're going to be talking about mapping the acoustic marine environment and using acoustics to map the marine environment. I wonder if for the sake of context, if you could take the time to just introduce yourself to the audience, please?
Chris:
Thank you very much for having me. Happy to be here. My name's Chris Verlinden, I'm am oceanographer, and acoustician. I'm originally from Portland, Oregon in the United States. I spent about 14 years in the coast guard as an officer, primarily serving on icebreakers in the Arctic. I spent the last few years of my career on loan to the Navy doing ocean acoustics research, everything from finding submarines to whales. I got out of the military about four or five years ago and helped found a small company called Applied Ocean Sciences, where I currently serve as the CTO. We do research in ocean acoustics and marine environmental protection.
Differncen between a Soundscape and a Soundshed
D:
I noticed during the introduction there, you refer to it as ocean acoustics. A lot of the time I hear people calling acoustic environments, soundscapes. For me it would make much more sense to call them soundsheds- the same way we talk about a watershed or a viewshed. I imagine that this acoustic environment has geographic limits. Could you explain to me why we don't often refer to this as being a soundshed?
C:
I actually love that question because I'm secretly starting the movement to start the term soundshed. I’m referring to the study of ocean acoustics, which is really just the study of sound in seawater and how sound propagates, its impact on the environment, and how you can use sound to sense the environment. When I use the word soundscape, I think of it as a way to describe everything that you'd hear in the ocean.
It's sort of the cacophony of man-made sounds from ships in industry, oil exploration, the sound of biologics like marine mammals, fish, snapping shrimp, flocking fish, things like that. As well as natural sounds, everything from breaking ice, breaking waves, and wind and seismic activity like earthquakes and volcanoes.
I think of soundscape the same way I think of a landscape. In a landscape, if you're standing in an environment, is everything that you can see. Well, the soundscape is everything that you can hear.
I think of the term like viewshed or watershed. A watershed is an area where all the water in the area flows to a single point or, a viewshed is all the places that I can see from a point. I think an appropriate use of the term soundshed might be to refer to the sound footprint from a single source, or maybe the sound footprint of a single receiver. Practically, if I have a microphone or an underwater microphone called a hydrophone in this location, what are all of the things I can hear?
If I look at the footprint of everything I could hear, that would be the soundshed of that microphone.
Similarly, if there was a whale, let's say a blue whale calling at its usual noise level and frequency, there would be a certain area over which you could hear that whale. That would be the whale’s soundshed. The soundscape would be the full abundance of sound in the ocean. It's the cumulation of all of the various sounds that you're hearing.
A soundscape might not have a clear definition because, depending on the time or what is happening or what the dominant source of noise is, the variability of the soundscape will change. For example, if a hurricane is moving through the ocean or within a few 100 miles, all you're going to be able to hear is breaking waves and wind and the noise from the hurricane. Whereas similarly, if you're in an area near a shipping port, all you're going to be able to hear is these ships, but that could change.
Maybe the port closes at night, and over the course of the day that soundscape changes. So to me soundscape describes all of the sounds that you can hear in a given area and those geographic boundaries will change and shift over time.
Mapping using Active and Passive Acoustics
D:
When you were describing soundscapes, you were talking about all of the different noises you could hear. You also mentioned this idea of using sound to sense the environment. Oftentimes when we think about remote sensing from satellites we'll think about active and passive sensors. When we think about active sensors and sensing the marine environment, people may be familiar with using acoustic measurements to measure bathymetry. Could you give us a few examples of passive sources of acoustics that we can use to measure things, and what we can measure with them?
C:
This is a question I've devoted my life to, so I get really excited, I think it's a pretty interesting topic. First of all, just like with remote sensing from space where you have active sensors like synthetic-aperture radar (SAR) and passive sensors just collecting imagery, you have that same paradigm in the ocean.
Fundamentally, when you're studying the ocean, you have to use sound because sound is how we see underwater.
What I mean by that is all the electromagnetic radiation, all the visual band optics. The light just doesn't propagate well underwater, and neither do most frequencies of radio that we can use for communications. What that means is that if we want to image the bottom of the ocean, whereas on land, we could use radar or LiDAR- all the DARS. DARS don't work underwater. We pretty much have to use sound to image the environment. Similarly, marine mammals use sound to navigate, find mates, communicate.
If you want to communicate, if you want to see and sense your environment, if you want to navigate, sound really is your only option. Just like in space and air where you have passive and active sensors to do all of those things, similarly underwater, you have active and passive acoustics.
Active acoustics would be where you transmit a sound and use the way that that sound interacts with the environment to learn something about your environment. Just like you pointed out, the real quintessential example of that is bottom mapping sonars or multi-beam sonars, bottom-penetrating sonars. Where you put out a pulse of sound, you wait for the reflection off of the bottom of the ocean and sub bottom layers. You can also image what's underneath the sea floor. You use the travel time of that reflection back to your sensor to determine the distance to those things. If you're determining the distance to all things in all directions, you start to construct an image of your environment.
Active acoustics are a very effective way to measure certain things in the environment, but there are still downsides. One is that some of these active acoustic techniques can be damaging to the environment. It can be harmful to marine mammals and fish that you sound to navigate. In this new paradigm of swarms of autonomous vehicles exploring the ocean, every watt of power matters. If you have to put out active of pulses of sound, you're going to kill your battery a lot quicker.
Passive sensing techniques have the advantage of being far less energetically expensive, and the equipment is far less expensive too. The easiest example is passive sonar compared to active sonar. If I'm looking for a submarine with active sonar, I put out a ping. I wait for that ping to reflect off of objects such as submarines. If I want to define that submarine passively, I just have to listen very, very carefully for the sound that submarine makes and then use triangulation, with my array of hydrophones to determine the location of the source. Passive acoustic sensing isn't limited to just hearing whale song or finding submarines. You can do a lot of really cool things passively.
There was a really wonderful paper written in the nineties by a scientist at Scripps, Mike Buckingham. He was actually on my committee and he's a brilliant and wonderful guy. He wrote a paper called Acoustic Daylight. What he did was he drew a comparison to the way that we use our eyes to see and sense our environment in air to what we might be able to do with acoustics underwater.
Think about the way that we interact with and that we sense our environment around us above water. We use our eyes. Our eyes aren't active sensors, we don't have laser beams shooting out of our eyes like LiDAR measuring reflections off of everything. We just take the ambient electromagnetic radiation or light that's reflecting off of objects and scattering all around us. We receive that light on arrays (our eyeballs) and do some very clever signal processing in our brain. We are able to reconstruct an image of what the world around us looks like from nothing but back-scattered ambient light.
Michael Buckingham wrote this paper of how we might be able to use acoustics to do the same thing. He used noise from things like breaking waves on a pier to image objects underwater. It's a non-trivial problem, it’s very challenging to do. He opened the door for a lot of really interesting work in using passive acoustics to sense the environment.
We have used noise from ships to measure the temperature and salinity, and possibly even the pH of the ocean using the way that the temperature, salinity, and pH of the ocean affects how sound propagates through seawater. This can help with all kinds of things, from climate and weather forecasting to military operations, to understanding ecosystem dynamics.
Another really interesting example, Dr. Aaron Thode wrote a paper a while back where he used blue whale calls to measure seawater and bottom properties of the ocean.
Creating Underwater sensor networks
D:
You mentioned this idea of using an array of sensors to gather this information. You talked about it being a non-trivial problem. When you were talking about using the acoustics from a ship to measure seawater temperature, what kind of sensor array do we need to have in place in order for that to make sense?
C:
I'll let you know when I figure that out, it's a non-trivial problem and there are a lot of really bright people working on it. In fact, I have a current ongoing research program with the Office of Naval Research studying exactly that problem, which is what sensors and what combination, what configuration do we need in order to make use of those types of sources to measure the ocean?
What I want to do is I want to create algorithms that can take an arbitrary system of hydrophones or sensors anywhere in the ocean, and quantify how well we can use those sensors to measure the ocean.
You're not going to pursue a new remote sensing technique unless you really understand how well it's going to perform. Some of the mathematics that go into inverting ocean profiles and seawater properties using noise from ships are not non-trivial because you don't know exactly where that ship is. You need to localize the ship. You need to account for any type of timing or position offsets in both your sensors and the receiver, and you need to measure the properties of the ocean all in one big mathematical calculation.
There's a whole field of mathematics that really center on how you can cheat in that problem by using prior information. Things like AIS ship tracking data and the fact that your array is only so long, so your hydrophones can only vary in certain positions. I've really made a career out of finding clever ways to cheat in order to do more with less information on measuring the ocean with sound. I'm currently working on trying to find ways to determine the information content of sound on arrays of hydrophones so that we can use that to answer your exact question of what exactly we need to measure the environment. The answer is as many hydrophones as possible, as perfectly surveyed as possible, with as quality of synchronization as possible. How good is good enough and how many hydrophones do you really need? That is a really an open question and something that we're hoping to answer.
Getting Good GIS Data from Aquatic Sensors
D:
I heard you mention AIS data. AIS data is this broadcast that comes off ships and it gives you speed, direction and a little bit more information about the actual ship and its position. Can we use that and say, "Oh, we know that ships travel along these lanes here," and put an array of hydrophones along those shipping lanes and start doing something there? We know where the ship was, we know the size of the ship. We can make some assumptions about how much noise it was producing with an array of hydrophones that hopefully weren't moving. Is that a feasible way forward when we think about using these passive sources to map the marine environment?
C:
That's something that I've been wanting to do for ages. We've got colleagues at University of Hawaii who've been thinking about that same thing for a long time as well. Instrumenting underwater cables and things that are already on the bottom of the ocean in areas where you have access to sources of sound like ships in order to do exactly that.
The advantage of fixed sensors is you only need to survey their location once and they don't move. There's a lot of advantages to those types of fixed sensors. So yes, I think 100% the more hydrophones that are out there in the ocean the better. You shouldn't put them out randomly. There will be places where we have more data, like next to a shipping lane. Then you also have to think about places where we really need those measurements. There are a lot of places where we just kind of know what the ocean looks like, either because we have a lot of sensors already in place or because the variability of the ocean in those areas isn't so significant.
I've done a lot of acoustic experiments off the Pacific Northwest. And I could probably draw for you what the temperature and salinity profiles look like 500 miles off of The Olympic Peninsula in Washington state right now, and be within a degree or two. The variability is very predictable and very well understood there. So with a couple sea surface temperature measurements, a few floats, a good oceanographer can tell you really what going on with the dynamics in that area fairly accurately.
If you look at a place like the north Atlantic Ocean off of the UK, for example, that is a complicated ocean. If you take a measurement in one place and you take a measurement 10 kilometers away, I don't know what they're going to look like, but I guarantee they're going to look different. There are a lot of really complicated oceanographic processes, deep convection sites, where water sinks in from the ocean bottom, and deep water masses spread throughout the ocean at ease. Currents meander hundreds of kilometers on a time scale of a few days. Oceans like that are very important to get more measurements from.
The real holy grail of ocean measurements right now is the Arctic. The Arctic is for much of the year covered with ice, and that makes it very difficult to get research vessels and sensing platforms that we have access to under the ice. The Arctic is one of the most critically important places to measure because it's one of the places that's changing the fastest due to climate change. This means acoustic measurements for measuring the ocean properties under the Arctic are incredibly important and challenging, because there are not a lot of ships in the Arctic. If you need to know as much as you can about the sensors and the environment in order to use those sensors to make meaningful measurements of the environment, well, the Arctic where there are a lot of unknowns is a challenging place to work.
Your question was a really good one. "If we just put these fixed sensors where we know exactly where they are all over the ocean, can we just use all the sound sources that are already in the ocean to map and measure ocean properties?"
Yes, but there will be places where that's not practical. That's why I think we need to do a lot more work on using moving platforms and sensors that might not be perfect. There was an interesting study done by DARPA here in the United States recently called Ocean of Things, where they wanted to put tens of thousands of biodegradable floats out in the ocean and instrument them with sensors, including potentially hydrophones, where these things are just drifting around and moving all over the place. That creates challenges because you need to really get the most accurate position of those sensors, you need to get timing synchronized perfectly. If you can do that, you could potentially use all of those sensors like the cube sats, but in the ocean.
There is a lot of really exciting work going on in this area. There's a company I love to work with called Subsea 7 that's making this autonomous sailing vessel that's a fraction of the cost of anything else that's out there. You could put hundreds, if not thousands of them out all over the ocean and position them optimally in order to get the best sample measurements of the ocean everywhere. It is a silent sensor because it's a sailing vessel and its semi-submersible nature makes it a really quiet platform. You could also instrument platforms that are already out there, like whales.
I used to teach oceanography at the Coast Guard Academy and I'd always start off with the description of why it's so important to get these measurements. I always use El Niño as the example. El Niño is a phenomena where warm water from the Western Pacific slashes across the ocean into the Eastern Pacific. It creates a big pool of warm water, which causes air to rise, and precipitation which disrupts global weather patterns. It can cause floods, famine, disruption to crop cycles, and disruption in ecosystems all over the world.
It can put thousands of people out of work, it can cause starvation, billions of dollars in economic disruption, thousands of lives impacted, people dying from the impact of El Niño. It is dynamics that we understand, we get geophysical fluid dynamics, we get how water moves and yet we cannot predict El Niño. This is a solvable problem in our time. If we understand the ocean better, if we make better measurements, if we make advances to ocean modeling techniques and ocean sensing techniques, including the acoustics ones we're discussing today, it's achievable to do things like predict El Niño.
We can save lives. We can preserve the economy. These are solvable problems in our time if we can just make better measurements of the ocean.
Using Sea Creatures as Spatial Sensors
D:
You're so enthusiastic about the problems that you're trying to solve. You said a couple things that you need to go back and clarify for me. You talked about the ocean of things, and it sounded like you were talking about using marine animals to crowdsource this mapping work that needs to be done. Maybe putting sensors on whale. Can you tell me more about that please? Earlier in the conversation, you talked about using whale’s acoustics to passively sense the environment. Are we then talking about putting a sensor on it and using a whale like a cube sat?
C:
First of all, DARPA’s Ocean of Things- it was a program that they created in order to see if you could make tons of measurements of the ocean in order to resolve all the things that we just talked about. If you could instrument the ocean to that degree, you could do things like predict El Niño. You could track where whales are in order to more accurately predict and mitigate human impacts on ecosystems. Having more sensors in the ocean is critically important.
Sensors, electronics, data exfiltration, these things have all gotten cheaper and more available over the years. It used to be that if you wanted a temperature sensor in the ocean, you better have 70 grand. Now there's an Open CTD project. It is a conductivity, temperature, and depth sensor. You can make it for a couple hundred dollars worth of parts and a PVC pipe. There's access to technology that we never had before. There are hacker websites, there's Arduinos, Raspberry Pis, Odroids, Teensys. Similarly, there is the Iridium satellite constellation so you can get data off of any of these instruments.
There is Starlink of course, and Kepler Space, which is a really exciting company out of Canada that's making a really nice communications constellation. The ability to interact with low-powered, low-cost sensors all over the world has never existed before, and it does now. I think in oceanography, we're entering into a new paradigm. It's kind of a revolution right now.
The space industry I would say is still about 10 or 20 years ahead of us. In space, it used to be that there were are a few dozen satellites. They cost billions of dollars, they were the size of school buses, and they had really nice cameras or sensors, but you only had a few of them. Now we have tens of thousands of sensors in communications hubs. Each of those cameras might not be as good, but when you combine the information from tens of thousands of lower end, lower cost, lower size platforms, you get a lot more information. We just got that memo in oceanography, and we just realized that having more sensors is good and having more sensors needn't be expensive.
I think it's established that having lots of sensors, even if they're poor quality sensors that don’t report as often, or in the case of whales, have complete control over where they go, are still a really good thing. So now let's talk a little bit about what would happen if you instrumented a whale or an elephant seal with a sensor. I say elephant seals, because they're one of the deepest diving marine mammals. They have a higher blood oxygen content and a higher concentration of hemoglobin in their blood than any other animal I'm aware of. They can stay down forever.
Put a temperature sensor, a CTD, like we just talked about on elephant seal, you'll get really incredible profiles of the ocean. You can listen to what the animal is doing. It is really important to understand the way that animals interact with their environment, like how whales use acoustics. Just the biological information we could gain from that would be incredibly valuable. If we can localize those animals using the acoustics around us and other technologies, perhaps we can analyze across groups of animals just like an array of hydrophones.
I think there are two really critical things to understand here and that's that you can use ambient noise to measure the environment. You could use the noise of ships like a sonar to localize things underwater or to measure temperature, salinity, things like that. You can also use the differences in ambient noise, from things like ships and waves recorded on groups of hydrophones to measure how much time it took for the sound to travel between those two instruments. Temperature and salinity affect the speed of sound, and that can give you estimates of the temperature and salinity of the ocean. You can also use passive acoustics of sources like whales and ships to measure the ocean bottom’s location, shape, and material.
A good example of that is a paper by Martin Siderius at Portland State who created something called the passive fathometer. Essentially, he takes an array of hydrophones, no sources of sound anywhere, and he does something called beamforming, which is a signal processing technique that allows you to listen in just one direction. He beamforms up and he beamforms down. "All the energy coming from the sea floor is coming from the surface, just reflected off the sea floor, because all the things that make sound are at the surface." "The time difference between the up beam and the down beam has to be the travel time between the hydrophone array in the bottom and back up.” So just like an active sonar, he's able to use the ambient noise of breaking waves and things at the surface in order to image the bottom. It's not inconceivable to think you could do passive multi-beams, or passive sub-bottom profilers. Martin and I have actually written papers about measuring bottom loss, which can tell you what the bottom's made out of.
If you have passive hydrophones anywhere and everywhere you can get them, on floats, on animals, or fixed platforms all over the ocean, you can make very meaningful measurements of both the soundscape and the environment, including temperature, salinity, pH and bottom properties. This matters for ecosystem dynamics, economics, oil and gas exploration, national defense, etc. This stuff matters, and I do think having hydrophones everywhere, including potentially on biological cube sats would be really beneficial.
Comparing Earth Observation From Space To Remotely Sensing The Deep Ocean
D:
This is a real eyeopener for me, this idea of using biological cubesats as you call them. You were referring to the idea of using groups of these in a mesh network to image and sense our environment. You started making those comparisons to what's happening in the space industry at the moment. Why do you think we are pouring money into space and earth observation platforms as opposed to earth observation platforms in the water? Is it more difficult to put things in the water than in space?
C:
I apologize if I offend any of my space colleagues now, but yeah, I think that's exactly why. Don't get me wrong, the assets we're putting in space are very valuable. This cube sat revolution has changed the world in a positive way. That being said, those space guys just need to stop complaining. When they put something into space, they act like it's this engineering marvel. They've got one atmosphere of pressure to deal with between the inside of a platform and the outside of the platform. Try putting something on the bottom of the ocean, where you have hundreds of atmospheres of pressure. Everything needs to be perfect to millimeters, otherwise it'll literally explode or implode. We're putting our sensors in acidic baths of some of the most corrosive stuff known to being- salt water. Yes, space is sexy. Space is important. Space-based sensors and space-based communication assets can really improve our lives, but understanding the ocean is critically important as well. It is very challenging and it's very expensive.
You hear people say things like, "We know more about the surface of Mars than we do the floor of the ocean on our own planet." I once wanted to examine how true that statement was. I was hosting a panel for the American Geographical Society between Don Walsh, Bob Ballard, Sylvia Earle, and Kathryn Sullivan, four pillars in the ocean exploration community, and we had this exact discussion. I asked them, "To what degree is it true that we know more about the surface of Mars than we know about the sea floor on our own planet?" And it really depends on what you mean.
We have a bathymetry map of the whole world. There aren't places where we know nothing because of the space people. Satellite-derived gravity measurements, gravitational anomalies, the Shuttle Radar Topography Mission, because of things like that we've been able to map out with at least very poor resolution everywhere on the surface of the earth. If you want to talk about resolution that's good enough for things like seismic exploration, or understanding ecosystem parameters, or where mineral deposits are or what the bottom's made out of well enough to be able to model all the acoustics in order to do things like find submarines or whales better, then you're talking about the need for maps of the seafloor with resolution better than 10 meters, and we don't have that. Depending on who you ask, I think it's between 5 and 15% of the ocean is mapped well enough to do that, whereas the entire surface of Mars is.
By at least some very meaningful metrics, we do know more about the surface of Mars than we do about the seafloor on our own planet, and I think that's problematic. There's a lot of really exciting efforts going on and I think Seabed 2030 is one where there's a lot of academic institutions, government programs, research labs, nonprofits, and companies that have combined together to try to create a comprehensive seafloor map of the entire world by 2030. It also represents this level of international cooperation, which I think is rare and incredibly powerful. We should use that as an example of ways that we can share data more effectively on a lot of different levels, acoustic data, temperature, salinity measurements, things like that.
Kathryn Sullivan, the former director of NOAA, the first person to have walked in space and gone to the bottom of the Mariana's Trench, a low key hero of mine. She once said, "We need to API the ocean." What she meant is we need to work on data availability, data sharing, just like those folks who are working on the seafloor map for 2030, and we need to do that for all of our ocean measurements.
I think that people are seeing that sensors are becoming more affordable. The problems that we can solve by understanding the oceans better are coming into our daily lives more. Things like understanding if storms are going to get worse or better with climate change, things like knowing which fisheries are healthy, or knowing which coral reefs are going to die and which sites might be viable for coral reefs in the future. People are starting to really understand how these things map to their daily lives. As a result, we're going to see a real increase in how much people care about and think about ocean exploration and ocean sensing in the future.
Understanding Marine Ecosystems
D:
I just want to try and make one more point here about this before we move on and perhaps talk about noise pollution and what that means for the ocean. You talked about the bathymetry model that we have for the deep oceans, and you were saying it's not good enough to accurately model the acoustic environment. Early on in the conversation, we were talking about how acoustics is the way we see underwater. Is that actually a limiting factor when we think about the bathymetry model that we have today? Is that the limiting factor when we think about seeing underwater, about sensing our marine environment?
C:
Yeah it is, think it this way, all over the world people are talking about 5G communications now. 5G communications could be very wonderful, very high bandwidth comms everywhere, but it's a challenging media to work with because 5G doesn't travel through objects. Heaven forbid that there are leaves in the way, so having really incredibly accurate maps of the landscape is really important for understanding this 5G communication infrastructure.
There's been a really sharp increase in the commercial, as well as government need for very accurate maps of the surface of the planet. In order to understand what you can communicate with, what you can see with acoustics underwater, you need to be able to understand what the bathymetry looks like.
Similarly, radio propagation depends on atmospheric properties. For certain things like long-range radio communications and over-the-horizon radar, you need to understand those things intimately.
However, for knowing whether or not you're going to get Wi-Fi somewhere, you don't necessarily need to pull out your temperature sensor and measure the humidity, but you do need to do that underwater. If you want to know where you can hear a humpback whale, you put out a hydrophone say, near a shipping lane because you want to be able to warn mariners if there are a lot of whales in the area so they can slow down. Nobody wants to hit a whale. If you want to know where your hydrophone can hear whales, you need to know the bathymetry perfectly. As well as you possibly can, within a few meters. You need to know that the depth accuracy everywhere, you need to know what the bottom is made of. You need to know if it's sand, is it rock? Is there sand with a little bit of rock underneath it?
You need to know the temperature and salinity of the ocean everywhere, because temperature and salinity affect the speed of sound, and the speed of sound can cause it to refract differently and bend back up to the surface or possibly down into the bottom. Finally, you even need to know the pH because the attenuation of sound is dependent on the pH. If you really want to understand your soundshed, going back to what we talked about earlier today, you need to know as much as possible about the ocean.
There's hope that if you have hydrophones out in the ocean, you can learn more about the ocean, and that improves your models. In order to do ocean sensing, we need to know the ocean as well as we possibly can. For example, whales when they interact with their environment, they're doing it acoustically. They can't see very far underwater, so if a beluga whale wants to know where it's food is, it's finding its food with acoustics. It knows its food likes ocean environments of certain types, maybe they hang out on ocean fronts, boundaries between warm and cold water, or maybe the food likes to live above ocean bottoms made of certain properties. Whales know what the sea floor is made out of. They know where ocean fronts are. They know how to navigate. They know how to find mates. They know how to find prey. They know how to find dangers in their environment.
Whales know a lot about their environment from what they're learning acoustically. We've really only scratched the surface of what we can do with acoustics. Yeah, we can find a submarine, a giant hunk of steel underwater, but we don't know where every single ocean front is everywhere all the time. We don't know what the bottom is made out of everywhere. We spend a lot more money than beluga whales do. Beluga whales keep me up at night and they should keep you up at night too.
I helped with a study by some researchers at Woods Hole, including a gentleman named Aaron Mooney, just a really brilliant scientist who was looking at how well beluga whales could direction find. One of the things they've found is that they can determine the direction sound is coming from, something like an order of magnitude, 10 times more precisely than they should be able to given the size of their sensing organ. They can defy modern mathematics in ways that we don't understand. If we can understand that a little bit better, then we can figure out how to use ocean acoustics to sense our environment a little bit better.
We can understand the environment better for its own sake. Understanding ecosystem dynamics, marine environmental protection- If you know where coral reefs are dying, you know where to focus. If you know the viability of the ocean for certain species, you know where to concentrate your efforts. There is just an abundance of discoveries to be made that have world-changing applications. To me, that's why studying sound in the ocean is so exciting right now.
Managing Underwater Noise Pollution
D:
I'd like to move on now and talk about mapping and measuring the sound in the marine environment. When we think about sound in the marine environment, a lot of us will think about pollution, noise pollution. My question is, how do we define noise pollution? Is it simply all anthropogenic noise that is put into that environment? Is it a certain frequency over a certain time period? Could you give us a working definition of noise pollution when we think about the marine environment?
C:
If you're in the European Union, you define noise pollution as ambient noise created by humans that can be harmful to the environment defined as marine organisms or anything else. We don't define underwater sound as a pollutant in the United States as of yet. The Noise Control Act of 1972 in the United States really only defines noise pollution as it affects human beings. Noise pollution for us is defined as creating sounds that are harmful to either creating stress or physical damage to human beings. Underwater sound where there aren't a lot of human beings is not currently classified as a pollutant in the United States. And I think that's going to change very soon. And I think we've already seen the beginnings of it.
There's been a lot of development lately in offshore wind, which I think is a really wonderful here in the United States. It's existed for quite a while in much of Europe, but we're really starting to hit it hard in the United States, which I think is a really good thing. These offshore wind developers have come from Europe, where they've already had to go through the process of determining what the environmental impact of these platforms is going to be. They've already got the procedures in place. They already have done studies to see how the sound from these offshore wind platforms could impact the marine environment. When they're coming to the United States, the Bureau of Ocean Energy Management is requiring them to do that same work. Even though noise isn't technically a pollutant here in the United States, they're really treating it as one in environmental impact assessments.
If you want to put in a new pier and you're going to do pile driving, pile driving's loud. You have to be able to determine what your impact is going to be on the environment and take necessary mitigation actions. I think that's a really good thing, and it's been forward-leaning to require that of the wind companies that are doing the work, and who are already prepared to do that work.
There's a wonderful company called Jasco based out of Canada, they've got a U.S. office as well. We used to compete with them, but we got tired of losing, so now we work with them as often as possible.
They do just a great job at understanding the environmental impact of industrial activity on the habitat on marine mammals. So if we take a step back again, you said, "How do you define noise pollution?" There are legal definitions, and that's what we just talked about, but there's just working definitions as well. I think underwater noise pollution will be sound made by human beings that impacts the environment. That can be more nuanced and more complicated than it sounds on the surface.
Considering the original example I gave of sound in air, it's pretty easy to determine if that's having a negative impact on human beings. Human beings are difficult animals to work with. That's why I became a physicist, but it is possible to ask a person, "How are you feeling?". It's possible to ask them to come in for regular tests so you can evaluate their health. You can determine if regular loud noises are causing stress hormones like cortisol and aldosterone to increase in their systems. It's possible to stick a camera in their ear or whatever doctors do to determine what sort of impact you've had on their hearing.
Whales are a little more difficult to get them to come in for regular appointments where you can ask them how they're feeling. Moreover, the way they use acoustics is so much more complicated and more nuanced than the way human beings use acoustics. It's hard to know how the sound that we produce impacts them.
We use sound to hear and it's about it. We use our ears to communicate with each other. Maybe find things generally, but the whales use sound for everything. Think about this, if I put you in a room with nails on a chalkboard, just constantly going for a month, the sound of nails on a chalkboard, you would lose your mind. Your stress hormones would be elevated, you'd lose sleep, and there would be significant health impacts on you. There is no physical reason why that should be true. The elevated levels of that sound, the frequencies, there is nothing physically about that sound and how it interacts with your ears drums that should mean that that's really painful for you. The only way that I know that nails on a chalkboard are painful for some people is because we can talk to people. We can't talk to whales about that. We don't know if there's an equivalent of nails on a chalkboard for a whale, because the way that they perceive sound is so much more nuanced and so much more complex than human beings.
That being said, we can do our best. I've recently been very lucky to participate in a study in collaboration with the World Wildlife Fund, and NOAA, here in the United States and Penn, a working group on the Arctic Council to predict future noise pollution levels in the Arctic from changes in shipping traffic. We ran some really incredible acoustic models that we're very proud of. We modeled every ship everywhere in the world all the time. Every minute, every day, every month, every year in order to create statistics on what the sound has looked like over the last 10 years and what it might look like over the next 10 years. We showed our results to our partners at the World Wildlife Fund and they were like, "Cool, nerds. What does that mean?" They didn't say that, they're very nice. What we realized is we had all these statistics and decibels and charts, but it didn't mean anything because it didn't tell us how that sound might actually impact the environment.
We asked one of our lead marine biologists, Dr. Kerri Seger, if she could put together some sort of a map by species. What she did is she took all the information that we had on the spatial extents for all of these critical species, so we had maps of where they were. Then she took all the information we had about the audible range of these species as best we could tell, and integrated the sound energy on their audible bands in the area where that animal lives. We were able to start coming up with risk scores by species and region that told us how much sound energy there would be.
Of course, there are differences. A gunshot or an explosion is a really intense sound that lasts for a second, but it can irreparably damage your ears and your eardrums. Similarly, if I played really loud, really bad music at 130 decibels for a month straight and never gave you any reprieve, that would have different physiological impacts on you. You would be stressed, you wouldn't be able to communicate, you wouldn't be able to talk. If you're a whale, you wouldn't be able to navigate, find food, find mates, etc. How you quantify how bad sound is for an organism has to be classified in different senses. There's a whole set of guidelines created by NOAA here in the United States, and there are similar guidelines in Europe and all over the world that help us understand what the impact of sound is on marine mammals.
One of my favorite studies was by a group called JAMA in Northern Europe, where they wanted to understand the impact of shipping noise on marine mammals in the North Sea. One of the statistics they reported on was the percentage of the time that sound was above certain thresholds that would be either damaging or uncomfortable to marine mammals. That is really important from a stress perspective, how often can this whale find mates, etc. Maybe if a ferry crosses through that whale's habitat once every few hours, it's not a big deal because they can still find mates, food, and communicate in between trips. If it's all the time though, that really matters a lot. So, to your original question, how do we define sound pollution? Well, it's man-made sound that affects the environment. But how do we decide what the impact of that pollution is on the environment? I think there's a lot of research that still needs to be done.
Bubbles as Sound Barriers
D:
I used to work for a company that was building offshore wind packs, and they were experimenting with this idea of using a bubble net, blowing bubbles up from the seafloor to protect the area that they were working in from letting too much noise escape from the point source. When you think about the future, is there a world where we noise proof our shipping lanes, noise proof our other activities? Can we set up barriers in the ocean? Can we use bubble nets? Is there any way to protect the marine environment from the noise that we are creating?
C:
We just submitted a grant this week to see about the viability of doing bubble curtains around ships. It's a pretty far-out idea. I would say a lot of experts in the field have said it would be very challenging. First, why might it work? Well, if you're doing pile-driving and let's say, you've determined that it's going to be really loud. You've determined that there's a lot of marine mammals in the area. You've done your homework and your due diligence and you don't want to impact those marine mammals. One of your options is to essentially lay hosing down on the bottom of the ocean. It's more complicated than this, but basically, it's a tube with a bunch of holes poked in it that you pump air through. It emits these big strings and curtains of bubbles around the pile-driving thing, and that blocks the sound. You might think to yourself, "Well, that seems silly. Why do bubbles block sound?".
If you've ever been in a swimming pool, could you hear the people talking above the swimming pool? Probably not. Similarly, if you were in a swimming pool and made a bunch of noise, the people outside the pool couldn't really hear you. That's because the pressure difference, density difference, and speed of sound difference between water and air is so great that sound waves see that boundary between air and water as almost like a perfect reflection surface. Very little energy can travel from water to air and vice versa. If you start putting all these little bubbles in the ocean, you create a wall where the sound reflects off that wall and it can't propagate through it. So in theory, bubble curtains can be a really effective mitigation strategy.
There's a researcher at The National Oceanography Center (NOC) in Southampton named Tim Leighton. We call him the bubble king. He wrote an 800-page book called The Acoustic Bubble, about the acoustics of bubbles. They are indescribably important. Bubble acoustics can cause sound to be trapped on the surface. It affects underwater communications, it affects everything. Bubble acoustics is an entire discipline of study that is really important for reasons like this.
To your question, can we just soundproof entire shipping lanes? Most people would just flat out say, "No," because it's going to be too expensive, shipping lanes are massive. But maybe just in places where it really matters. Maybe just in places where you have a shipping lane going through a critical habitat. Look at the Santa Barbara channel in the United States. LA Long Beach is the highest volume port in the United States, and for a lot of vessels, the most efficient route is to go through the Santa Barbara channel. It is a habitat for a ton of highly endangered marine mammals and other species that are sensitive to sound. If you're in a ship that operates often in areas where species might be really sensitive to sound, could you inject a bubble curtain around the whole of your ship? It would be very challenging, there are a lot of significant engineering challenges to be able to do something like that.
Icebreakers actually inject bubbles along the whole of their ship to reduce the friction with the ice. So there are systems, albeit a lot less complex, that do things that are similar. I don't think it's impossible, and there might be other ways to reduce the sound from ships. There are really good engineers out there who specialize in making things quiet. You would never accept a car that was so loud that you could never hear anybody while you were inside of it. That's because really smart engineers tried very hard to make those things quieter. I think there's probably a lot of gains to be had just purely in engineering of ships.
I forget the name of the company, but some folks have designed these sort of... They're not springs, but basically shock absorbers that you mount to things like generators and engines. That decouples the vibrations from the machinery with the whole of your ship, which prevents some of the acoustic emissions. There's a lot more that can be done. The Navy's been trying to make submarines quieter and quieter for going on 80 years now. Taking some of that engineering and applying it to commercial shipping, it'd be expensive, but I think there's a lot more work to be done there. As more and more people start to think of sound as a pollutant, there are going to be requirements that come out that make us take some of those engineering steps.
D:
We have come a long way in this conversation and I'm really enjoying it. This has been absolutely great. You're working on a ton of different things, you're exposed to a ton of different ideas. What's the thing out there that's got you most excited for this year and maybe for the next five years?
C:
Oh, man, that's a tough one. You might be able to tell, I get very excited about my work. I love oceanography and I love ocean acoustics for all the reasons we talked about. There are so many unsolved problems that are really, really important.
D:
Let me just stop you there. You talk about unsolved problems that are really important, and it sounds like you're excited by this. You're not overwhelmed by the challenges, you're excited by these unsolved problems?
C:
Yeah, of course I'm excited. I don't know about you, but if I do something that I think is impactful, it makes me happy, it makes me think I made a difference. If I can make tiny little advances in our understanding of underwater sound and of the ocean, I think that there are huge benefits to society and to the world. There are so many problems that I could contribute in a small way to solving that can make the world a better place, and that can help us understand how to protect whales better, help national defense, that can help us address climate change. The amount of unsolved problems that are solvable in our time and the impact those solutions would have to humanity is what gets me up in the morning. So to your question, there are two projects that have me most excited right now, one of them, because it's just so darn cool, and the other one, because it's so darn hard.
The one that is cool is the project I told you about already with the World Wildlife Fund and the Artic Council and NOAA. First of all, our government program manager is wonderful. She is a solving hard problem, she's pushing us hard to really do the best job possible to understand how shipping noise in the Arctic is likely to affect Arctic ecosystems in the future. We have just gone all out. We've taken the most sophisticated acoustic propagation models we've ever made and analyzed and optimized them for all sorts of computing architectures, to be able to simulate every ship, everywhere, all the time.
We've brought in marine biologists, psychologists, physicists, and atmospheric scientists to work with the World Wildlife Fund. The work we're doing is good. We've gotten to work with world experts in everything from economics who will drive how shipping will change, to some of the best ecologists and marine biologists there are, and department of transportation people who just get shipping traffic. The work is good, and it will be delivered by the World Wildlife Fund and these Arctic Council people to decision makers who will get to look at these maps of, "Oh, there's a shipping lane here and there's a critical habitat for this whale there." Maybe if we just moved the shipping lane, or maybe if we made this regulation or that regulation, it would have this huge impact on the future of conservation in that really critical environment. I think the combination of how good the work is, the fun people we get to work with, the potential impact the study has, makes it all really exciting to me.
The second project that has me really excited is one we've alluded to, it's a basic research program funded by the Office of Naval Research here in the United States. It's about the information content of ocean noise. It was inspired by work done by my PhD advisor, Dr. Bill Cooperman at Scripps a few years ago where we're trying very hard to understand everything you can learn about the ocean, from sound that's already in the ocean. Be it from ships, be it from whales, regardless of the source. How can we better understand what we can learn about the ocean from underwater sound?
I got a chance to collaborate with brilliant scientists at Scripps, at Georgia Tech and Woods Hole and Portland State, University of Washington, and in my own company, at Applied Ocean Sciences and it's so darn hard, the math, the work you need to do to try to synchronize all your clocks and position all your ships and your sensors and parameterize the ocean in really clever ways to reduce that mathematical search space. It's so darn hard and therefore very rewarding when we make progress. So yeah, those are the two most exciting things I'm working on, the ocean conservation, noise pollution stuff in the Arctic, because of the impact and the information content of ocean noise stuff for the Navy, because it's so hard and so interesting, and I also do think critically important.
D:
Chris, I really want to thank you for your time. I have thoroughly enjoyed talking with you. You're brilliant, you're enthusiastic, and you're human. This has been a really inspirational conversation for me. So I really appreciate it. Let’s say somebody else is listening to this and they think, "Well, who is this guy? How do I get in touch with him? How can I reach out to him?" Where would they go to do that?
C:
If anybody after this still to hear more, so the Applied Ocean Sciences website has contact info. It's appliedoceansciences.com. My email is chris.verlinden@appliedoceansciences.com. I'm always happy to talk acoustics with anyone and everyone. Feel free to cold call me or email me, I love talking about this stuff.
Our guest today is Danny Arribas-Bel, PhD. He is the Senior Lecturer of Data Science at the University of Liverpool, and Deputy Programme Director for Urban Analytics at the Alan Turing Institute. Although his focus now is in urban planning and development, he began his academic pursuits with a PhD in economics. Building this background in the economic driving factors behind cities and communities has allowed Danny valuable insight and context into why we see certain patterns in these spaces. Through the Alan Turing Institute, Danny contributes to the Urban Grammar project, a spatial data science project which aims to classify the world’s cities based on the form and function of their unique organizations.
What is Urban Grammar?Looking across the world, and across time, we have seen many cities rise and fall. While none of these cities are close to identical, they share an intriguing number of similarities in how they are built, populated, and utilized by their residents. Those who study the urban sciences, from planning to architecture to economics, have great interest in being able to quantify, qualify, and compare these places across the dimensions of time and space. This is where the Alan Turing Institute’s Urban Grammar project comes in.
In the same way that we can break down the grammar and syntax of a language, we can begin to break down the characteristics of cities. Before this can happen, however, there needs to be a more or less consistent dataset to pull from for analysis. Considering the characteristics of cities and their residents are constantly changing, it is a huge challenge to find data with the temporal granularity necessary to be useful to track changes.
The best and most complete data for Urban Grammar’s purposes are sources like the Census, or Ordnance Survey data. Official sources like this are rich with attributes, and reliable and consistent in their collection methods. The issue, of course, is that Census data is only collected every 10 years. In order to fill in some of the temporal gaps, researchers utilize spatial data generated from satellite imagery. This may take the form of crowdsourced OpenStreetMap datasets, which may come with built in attributes, or datasets generated through earth observation and artificial intelligence workflows.
Artificial intelligence (AI) practices like deep learning and machine learning are invaluable to Urban Grammar’s mission to make data available to study spatial changes in our urban landscapes. GIS workflows using AI generally consist of collecting and labeling a large amount of imagery in order to create training data samples. These samples are then fed into an algorithm to train it to begin to generate future samples and data on its own when provided similar imagery.
Again, the biggest challenge here is the availability of satellite imagery at a temporal scale that is useful to track change, and of high enough quality to perform deep learning workflows. Generally, this can be combated by combining data from many sources, including government provided satellite imagery, Census and Ordnance Survey data, and crowdsourced geospatial data.
Data Classification and SignaturesOnce the necessary data has been aggregated, it is time to move into the equally large task of analysis. As mentioned, analysis at the scale necessary to understand entire cities requires deep learning and artificial intelligence workflows in order to match the pace at which the underlying data becomes available. These workflows require, and have the ultimate goal of classifying data.
Considering the diversity of study areas researchers deal with, how are these classes created?
Well, first and foremost, Urban Grammer approaches categorizations of cities through the framework of what they call signatures. Signatures are the resulting classes of analyzing the form and function of different spaces using concepts related to morphometrics. Form in this context is what does the space “look like”. This can be the building footprints, street network, or significant natural features revealed in imagery. Essentially, if you were learning a language and viewing flashcards with pictures of different types of buildings and features, form would be what the structure “is”.
Function is the other side of the coin here. This is what the structure “does”. For example, while the form of a house is a building, the function of the house is a living space. The form of a highway may be a street, but the function of that street is transportation. Oftentimes, form and function go hand in hand, one follows the other. Form and function can be distilled into signatures through a plug and play sort of process using AI algorithms, fine tuning it for optimal I/O.
Considering urban spaces are unique and different from each other, the inputs and outputs of these algorithms will be similarly unique, and likely will not result in uniform classifications across the world. This is where humans enter the system, providing checks and “ground truthing” on the resulting data by adding the context and knowledge of human and spatial sciences.
Applying the Sciences to Spatial DataUnderstanding the function and form of the world’s cities is not a new concept. This has been a goal, direct or indirect, of many disciplines including architecture, urban planning and design, civil engineering, political science, archeology, GIS, history, etc.
If you have ever visited ruins on vacation, you have likely found yourself theorizing the day to day uses of different spaces on site through your modern lens. This can be a fun conversation starter with your traveling companions, and you may find yourself getting a bit creative with assigning applications to the space only to be surprised by an informational board that provides some correcting historical context.
Every dataset acts as a snapshot in time. Maps themselves become outdated as soon as they are put into print. By collecting, categorizing, and documenting datasets over time, researchers can build more nuanced pictures of the past, present, and future. A large part of this process for modern scientists is writing and utilizing code. Where once upon a time physicists used mathematical equations and the written word to document processes, now, researchers find code and the algorithms themselves to function as reusable and portable documentation for their research.
The building blocks of cities are pretty similar all around. Streets are streets and buildings are buildings. Every society has the same basic needs, food, lodging, work, recreation, etc. For this reason, AI practices like transfer learning, which promotes interoperability of algorithms by reusing the most base levels of code, then customizing it based on needs, are growing in popularity amongst data scientists.
Of course, these algorithms need to be adapted based on location to accommodate for differences in source data, and the technical and cultural variations inherently seen across our world. Adding regional context may add some steps to the process, but ultimately it results in richer, more relevant data for the area. Data may be beautiful, but it hardly exists just for the sake of looking pretty. We need data to make decisions. More locally relevant data results in better policy, and resource management decisions by those in charge. It allows for more nuanced predictions for the future, and a more acutely critical lens of the past.
A note from the author: If you are interested in learning more about how form and function have changed over time at the scale of day-to-day life, Bill Bryson’s book “At Home: A Short History of Private Life” does an excellent job of deconstructing the changes in use of our built world.
Our guest today is Matt Pietryszyn, the CEO and co-founder of Qwhery, a company looking to use smart devices to streamline interactions between citizens and their municipalities. After years in the public GIS sector, Patrician recognized a strain on 311 resources and wanted to help. As a tinkerer with a skillset in developing open data solutions, his response was Q11, a smart speaker application that is looking to improve ease of access to location information for everyday questions. Voice Assistants and Spatial Applications In the modern world, voice technology is becoming more and more commonplace. In 2019, 20% of Google searches were voice queries and in general, a hefty share of searched questions are users looking for information on things in their local area. To complement this, over one-third of Americans now own a smart speaker. Understanding the current landscape, it is unsurprising that some innovators are looking to give the people what they want, a smart speaker compatible application that will answer their spatial questions.
Smart speakers, such as Amazon’s Alexa, and Google Home, are interactive speakers that allow users to control their smart homes. This largely involves tasks such as turning lights on or off, playing music, or telling a silly joke on demand.
These platforms also offer app stores with custom libraries of functions for users to take advantage of. Amazon’s applications are called skills, where Google’s are called actions. Independent companies and developers can contribute to these app stores to add even more targeted functionality.
Although there are already a plethora of skills and actions on the market, smart voice assistance is still a very under explored space. Spatial queries in particular leave a lot to be desired, but hold a great amount of potential.
As smart devices store a user profile of home address, phone number, email, and a number of other preferences, the figurative distance to be traveled to start answering people’s spatial questions is shorter than one might think.
The trick lies in translating how humans ask questions, to how computers understand questions, then encouraging humans to continue to take advantage of this emerging technology as new capabilities are released. A significant part of this is porting web page based applications to voice based interactions.
Where a web map may be the driving force behind answering a user’s question, that simply cannot be communicated in the same way strictly through words. This requires rethinking how people interact with spatial information at its core. How Do Voice Skills and Actions Work? As everyone here may know, humans and computers speak different languages. Computer scientists have spent a long time navigating how to better translate between the two, and now we benefit from an extensive catalog of options. The people over at Q11 in particular are focused on adding geographical questions to this growing dictionary to allow you to ask “Alexa, when will my garbage be picked up?” or “Hey Google, what is going on downtown tonight?” and get back a quick and useful result.
Accomplishing this task is simple in theory, but takes time to implement correctly. The general workflow involves first converting the user’s question to a query, pushing it to the appropriate municipality’s API and backend, then returning an understandable response to the user.
Converting user questions into actionable queries has been the mission of smart speakers from the beginning. The process, called natural language processing (NLP) involves converting the spoken word into text, then extracting information from that text to build a query or command. The phrases people use for requests are called ‘utterances’.
Lists of synonyms are created for the words used in utterances. These lists are expanded in order to help train the program for the many language variations that the application will likely encounter from the public.
While applications like Q11 will respond to specific requests, for example “Alexa, what construction projects are there within 2 miles of me?”, it is also engineered to create queries from more general questions like “Hey Google, what construction projects are there near my house?”.
Once the utterance has become a query, it gets fed into the relevant locality’s API. As there is no set standard for how GIS services are administered, stored, and accessed, it is necessary for this API setup to be done manually using Qwhery.cloud by the municipality’s GIS or IT department. This allows them to take advantage of the potential features of the Q11 application, as long as they have a feature service available that matches the user’s needs.
Considering the discontinuity between various governments’s backend GIS systems, it is unlikely that we will see Amazon or Google themselves swoop in with their own custom spatial skills and actions.
It is simply more efficient for each organization to integrate their own APIs, REST endpoints, and services. When the localities themselves are involved in this step, it also allows them to better regionally adapt their queries and outputs. Where “nearby” may mean 10 minutes in a small town, it may mean an hour in a larger city. The Future of Spatial Voice Querying Voice assistants and artificial intelligence in general have come a long way very quickly, but of course, there is still ground to cover. We have already seen next gen features incorporated, like Q11 notifying users when the garbage truck should be expected, but there are both technical, and social challenges in the way of future uses of geospatial voice apps.
A broad goal that GIS departments have in implementing tools like voice and web applications is to reduce calls they get to 311 centers, and to generally produce better informed citizens. This mission began with geospatial web apps, and the issues began with promoting awareness of these resources to the target audience.
Geospatial voice tasks and actions face the same obscurity problem. If people are not aware they can ask Alexa what the zoning limitations of their property are, they probably will not think to. By the time they have researched the answer to their question, they may not have the need to ask it again in the future. This conundrum is why developers want to facilitate reusable content - questions that need to be asked over and over again. This produces returning customers who may be more willing to invest the time to explore additional capabilities.
A huge market for potential use of spatially-aware voice apps is commercial applications. A user might be able to ask “Alexa, what is there to do downtown tonight?” and get a number of events to attend. These can be sourced from the municipality’s Economic Development office, or even go down the path of sponsored messages.
Further opportunities in the private sector lie in integrations with things like the Yelp API, where a user could ask for recommendations of where to find the best pizza in their city or neighborhood.
Once the word gets out on the possibilities behind spatial voice queries, there will be no looking back.
Our guest today is Laura Duncanson PhD, an Assistant Professor for the Department of Geographical Sciences at the University of Maryland College Park. Since the beginning of her academic career, Laura’s eyes have been turned skyward to better understand and serve our planet. Her focus for years has been the user of Lidar technology to track the loss and growth of the world’s forests. In order to accomplish this, she has worked with the NASA Goddard Space Flight Center and a series of research partnerships, culminating in the Global Ecosystem Dynamics Investigation (GEDI) mission.
What is GEDI?
The Global Ecosystem Dynamics Investigation (GEDI) project, launched in 2018, is a spaceborne Lidar mission with the objective of scanning the world’s forests in 3D. Mounted on the International Space Station (ISS), GEDI maps forest height, cover, and canopy changes.
Of course, remote sensing scientists have been mapping forests with Lidar for years, what makes GEDI so groundbreaking? Well, one of the most important aspects of creating a scientifically useful aerial/satellite imagery dataset is the collection being taken from a fixed altitude. This is not too difficult to do for smaller areas.
Planes can be used at a national scale, and drones at a local scale, but until GEDI, there has not been a viable option for global scale Lidar.
Attachment to the ISS allows collections to be taken from a precisely known location, and for full Earth coverage to become a reality rather than an aspiration.
GEDI is equipped with three lasers, which produce eight tracks of data. Each laser covers a 25m circle on the ground.
These tracks are spaced 60m by 600m apart, covering a span of 4.2km on the ground.
This does, however, leave significant coverage gaps between tracks, requiring multiple passes to fill in the gaps. Considering GEDI is attached to the ISS, it is limited to the ISS’s orbit. This means the extent of taskability is limited to “rocking” the sensor from side to side to capture oblique angles of the target.
ALONG TRACK LIDAR RETURN ENERGY SHOWING VERTICAL DISTRIBUTION OF VEGETATION (Courtesy of UMD)
GEDI is a near infrared Full Waveform Lidar System (vs a Discrete Return Lidar System which people are more familiar with). Full waveform Lidar captures a more continuous reading of returned photons, allowing more detailed data at the cost of it being more resource-intensive to interpret. Surfaces which reflect more photons, like bulky canopies, result in a greater returned waveform, where narrower features, like small trees, result in a smaller waveform reading. An excellent animation of this process is provided here.
How is GEDI Useful?
The purpose of this highly detailed forest collection is for more than simply proving it can be done. Forest attributes like height and canopy cover can be used to quantify forest carbon content and change over time due to deforestation and afforestation.
Understanding the changes in carbon distribution in our world is vital for planning for the potential effects of climate change, and allows for monitoring real impacts of policies.
Although GEDI is state of the art technology, capable of collecting very high resolution data, it is not perfectly useful by itself.
Ground truthing and manual collections are still necessary to give complete meaning to the Lidar data.
The majority of forest carbon is sequestered in tree trunks. Given trunks are hidden under the tree’s canopy, they are very difficult to measure from the air, even with Lidar.
This means boots on the ground are necessary to collect training data on trunk measurements, which can then be combined with Lidar readings to extrapolate expected carbon values associated with different forest structures.
The GEDI project has a network of international research partners who help to provide training datasets for their local forests. There are still areas, notably Central Asia, West Africa, and southern South America that lack any on the ground measurements. They also lack data from aerial Lidar missions (planes) to act as checks against the GEDI data.
The best solution until this collection becomes more feasible is to collect this data in ecologically similar areas, then use it to infer values in the underserved areas.
The Future of GEDI
As futuristic as GEDI is, all of our forest data collection problems have not yet been solved. Being a near infrared system, GEDI is not capable of seeing through clouds, nor does it pass over the Poles. This, combined with the large gaps between collected tracks means it takes quite a while to get full wall to wall Lidar coverage of the Earth’s landmasses.
GEDI has been planned to run until 2023, however, the hardware is predicted to last up to a decade.
Researchers are hoping to appeal to the proper authorities to have the mission extended.
This becomes an even more noble pursuit considering NASA’s plans to launch ISRO, a Synthetic Aperture Radar mission (NISAR) in 2023. There are untold possibilities when it comes to being able to combine data products from both Lidar, SAR, and standard imagery collections. Losing the Lidar component to this would be a tragedy, especially considering the current GEDI mission is a result of over 30 years of campaigning and research.
Raw data is flashy, but ultimately of little use if the right tools do not exist to manipulate and derive meaning from it.
GEDI’s use of the less-popularized Full Waveform Lidar means that there is less existing data infrastructure than for discrete Lidar data formats.
Researchers and data scientists need to develop their own custom tools to capitalize on this information rich data. These currently take the form of Python and R packages. The NASA Data Center even provides some Juptyter notebooks, and the USGS provides LP DAAC for working with GEDI data. Although these obstacles cannot be overlooked, it means there is huge potential for growth in workflows and technology in the future as GEDI products become more widely integrated to existing and emerging systems.
GEDI data is available for free download here, and a key to the various levels of data products provided can be found here.
Microsoft's Planetary Computer
Our guest today is Rob Emanuele, a Geospatial Architect at Microsoft who is helping build their Planetary Computer. Beginning his journey with a Mathematics degree from Rutgers, he rose through the ranks of several computer engineering roles before working on the Geotrellis, and Raster Vision projects for many years with Azavea. This exposed him to the hands-on application of big raster data solutions with the Earth’s systems. He continues to support his open source roots by carrying the same group effort mentality over to the Planetary Computer project with Microsoft.
What is the Planetary Computer?
The Planetary Computer is a highly modular geospatial project by Microsoft that allows users to query, access, and analyze environmental data at global scales, taking advantage of cloud and blob streaming technology. In a space where many people still download data for their study area, it is exciting to see such an investment into heavy-duty GIS cloud technology. Cloud efficiency and power means unprecedented processing capabilities, allowing the Planetary Computer to help answer some of the world’s biggest questions.
From the beginning,
the Planetary Computer has been designed to integrate with open source elements, and to be worked into existing environments, especially Azure.
This open source integration is facilitated through tools in the Planetary Computer Hub. APIs are available through the Hub, as well as standalone, to promote easier querying of the many datasets being hosted in a consistent and efficient way. Combined with analysis ready data formats hosted by Microsoft, there is a lot of power at the user’s disposal.
The Planetary Computer hosts data from Landsat, Sentinel, ASTER, ALOS, GOES-R, NAIP, and many more geospatial data providers. These data are organized and made interoperable via their metadata. The project utilizes the Spatiotemporal Asset Catalog (STAC) to scrape these massive amounts of data for the information needed to integrate them with each other and promote a seamless use of data. Microsoft, however, does their best to leave these data true to form, recognizing the scientific value and meaning imparted by the preparation from their original sources.
Accessing and utilizing the tools, data, and services associated with the Planetary Computer is free. In order to get started, one must apply to participate in the preview, or beta program, as
not every component of the project is fully publicly available yet.
Once approved, you can generate an anonymous token which provides access to the data catalog, and can then be regenerated once it has expired.
The Components of the Planetary Computer
The immense scale of the data behind the Planetary Computer requires an equally flexible and scalable architecture to facilitate the flow necessary to accomplish the massive workflows it was designed for. Naturally, Microsoft is prepared for this due to their vast experiences gained through Azure, their cloud computing, data storage, and analysis platform.
The four components which make up the Planetary Computer are the Data Catalog, the APIs, the Hub, and the Applications built on top of the platform.
The Data Catalog contains petabytes of imagery, sourced from their original providers, then hosted together on the Azure cloud. The APIs take advantage of the standardized metadata produced through STAC to facilitate querying data of interest. The Hub is an environment, deployed via JupyterHub and preloaded with popular geospatial tools, allowing scientists to get to work quickly. The associated Applications are created by third parties and built on top of Planetary Computer resources. Combined, these components can attain the trifecta of hosting, searching, and computing data.
One of the ways Microsoft has been able to strategize for success with such a massive project, is being able to store the data immediately next to where it will be processed. This is one of the many benefits of having enormous data stores and servers at your disposal, as well as the resources to integrate Kubernetes clusters. The close physical proximity means the most cumbersome part of the process is the user's get request.
In terms of how the services are being delivered, cloud optimized GeoTIFFs (COGs) are the most common standard. Data, especially weather data, can also take the form of cloud optimized NetCDFs, HDFs, and Zarr data. These N-dimensional array formats are easier to “chunk” into more manageable sections. The ability to break up binary large object (BLOB) data is essential for efficiently streaming it. Partitioning the data, and describing it well via its metadata, allows APIs to index, query, and stream just the most relevant bits of data to the analysis, rather than pulling everything like we see with traditional downloads.
The components for the Planetary Computer are available on GitHub.
The Planetary Computer’s History and Future
The Planetary Computer did not come purely from Microsoft’s goodwill. They approached the beginnings of this project by acknowledging that,
In order for Microsoft to do well, the Earth must do well.
The threats of climate change, loss of biodiversity, and many other of the world’s problems would be bad for business. In an effort to set everyone up for success, Microsoft decided to leverage its resources to create an ultimate resource for earth scientists by pulling everything they need for interpretation into one place.
An all encompassing imagery analysis hub is not a new concept. Google Earth Engine notably was first on the scene. Esri’s imagery services and products have also been around for sometime in the pay to play realm.
One of the biggest shifts with the Planetary Computer will be the ability to keep costs low, to non-existent
for the full data pipeline with conscious integration into existing systems. Although they are very much competing for the same market, Google Earth Engine and the Planetary Computer have a collaborative and friendly relationship. Ultimately, the goal is to better the planet, and no one achieves such a task alone.
In the meantime, there are still obstacles to overcome before the Planetary Computer (note, another PC by Microsoft) will be fully up and running. STAC has done fantastic work in standardizing data categorization by utilizing a common JSON format for metadata, but data fusion is still a tricky and intensive process to sort out. Sensors from different manufacturers, and varied time intervals and resolutions provide challenges for true interoperability of data.
In the future, once the kinks have been worked out of the larger system,
There is tremendous potential for artificial intelligence applications.
We may see pre-packaged highly specialized models that users can run on open data, and seamlessly host and display the results in the cloud. In fact, we have already seen some of this with the Microsoft Building Footprints dataset. There are many industries that have questions about the world, and with time, we may find it is no longer on the distant horizon, but rather at our fingertips to solve them.
Recommended Podcast Episodes
Our guest today is Jeff Masek, the Landsat 9 Project Scientist for NASA at the Goddard Space Flight Center. Jeff’s role is to ensure the scientific integrity of the Landsat mission. This involves overseeing adherence to a number of project requirements, as well as investigating and testing the performance of the instruments onboard the satellite. He also has the privileged role of reviewing the images that come down from the mission. Jeff has been with NASA for over 20 years, applying his research skills obtained from Haverford College, and Cornell University. What is the Landsat Mission? Landsat is the longest running live land remote sensing system in the world. It is responsible for nine satellites which have collected over nine million images of our Earth’s landscape in the past 50 years. A joint mission between NASA and the USGS, Landsat has inspired similar missions abroad, such as the Copernicus Sentinel 2, the European Space Agency’s own project.
Landsat 8, and now Landsat 9, follow a sun-synchronous orbit, chasing the horizon to capture routine coverage of the Earth’s land surfaces every 16 days. If they work in conjunction as a constellation, this timeframe is reduced to full coverage every 8 days. Images are collected at 30x30m resolution, about the size of a baseball diamond. Although this is not the highest resolution on the market, it serves its audience well for the purpose of monitoring changes in the land due to the regular frequency of collection, and the general breadth of the archives across the near history of the Earth.
Landsat data is available to the general public via the USGS EarthExplorer interactive map. As a public good, this well-maintained raster data can be accessed by anybody, but it is generally most useful to earth scientists. The open dataset allows monitoring of natural resources, like agricultural fields, forests, and glaciers.
It is difficult to manage what you can’t measure, and thus Landsat data serves as a wonderful tool for more conscientious management of resources. In agriculture, it helps provide insight to crop health and yields, facilitating food security discussions. In forestry, Landsat imagery can be used to measure afforestation and deforestation, as well as to help monitor the effects of diseases or insects. These two areas in particular are vital for tracking changes related to carbon sinks to better understand impending risks from climate change. On that note, Landsat data has gained traction in the cryospheric community, allowing researchers to monitor changes in glaciers, ice caps, and other frozen water bodies over time. How Does Landsat Collect Imagery? Landsat 8 collects data across 11 bands in the visible light spectrum, near infrared, short wave infrared, and thermal infrared, as well as a panchromatic band. Landsat 8 and 9 use the Operational Land Imager (OLI) and the Thermal Infrared Sensor (TIRS) to fulfill the mission of full coverage every 16 days.
Considering there is a 50 year archive of data collected under the Landsat mission, great care is taken to promote interoperability between missions to avoid leaving any gaps in functionality. In fact, NASA and the USGS communicate with the European Space Agency to encourage standardization wherever possible, such as with ground correction and orthorectification, to create more scientifically viable products for our increasingly globalized research community. This is best achieved by using a common DTM to allow consistency in the supporting geodesic workflows.
Landsat satellites receive what is essentially a to-do list everyday which guides its collection. In special circumstances, such as natural disasters, the satellite can be tasked and has the ability to roll off nadir to capture adjacent areas of interest. A satellite can hold about a day’s worth of data at a time. This data is beamed down to ground stations whenever the satellite is within range. There are four primary stations. One is in Alaska, one in Norway, in Australia, and another in Sioux Falls, South Dakota at the EROS data processing center. These locations are closer to the poles to allow the longest possible communication between the stations and the satellites due to how the orbits line up. Before the USGS can begin implementing correction algorithms and harmonizing the spectral bands of Landsat products, they must first get the satellite up into orbit. This is where NASA comes in. They are responsible for building, and launching the satellites before turning over the reins to the USGS to collect, distribute and archive the data. It takes anywhere from 5 to 7 years to get a satellite into orbit from the initial inception of the project.
Testing alone can take up to a year. It is necessary to test at the component, instrument, and observatory level to ensure the integrity of the overall mission. These elements are tested in a vacuum chamber to simulate the space environment, as well as the hot and cold extremes, and even electromagnetic and soundwave testing to prove durability.
Considering it is a priority to minimize moving parts in the satellite construction, there are very few of these to worry about. The true limiting factor to the longevity of a Landsat mission is its fuel. Due to the slight gravitational pull of the Earth on the satellite while it is in orbit, that orbit will begin to degrade as the satellite is pulled closer. It is necessary to fire the thrusters occasionally to give the station a little push back out into its expected orbit. Landsat 7 has run out of fuel, and drifted close enough to Earth that its data is scientifically unusable. In order to be decommissioned, a reserved amount of fuel will be used to slow it down, and allow it to reenter the Earth’s atmosphere in a controlled and more predictable way to minimize risk to those on the ground. What Next for Landsat? Landsat NeXt There are a few challenges that the Landsat mission is looking to tackle in the somewhat near future. As mentioned before, the biggest dictator of mission longevity is fuel supply. This has birthed OSAM-1, a refueling and servicing special project. OSAM-1 would essentially be a robotic arm and a gas can, allowing existing Landsat projects to be revitalized for continued collection.
The wishlist item of current USGS Landsat customers is to have an image available for everyday. Although this target is unrealistic due to budget and resource constraints, it does provide a motivational target for the Landsat mission, especially in collaboration with other projects just as Sentinel. The Landsat mission and the ESA will likely continue to work together more closely in the future to enhance their shared scientific interests.
The most exciting future development of the Landsat program is Landsat NeXt. Landsat NeXt would be capable of collecting 25 spectral bands, creating unprecedented data access for the global community. The project is still in the design phases, but it is expected to be active by 2029. There has been inspiration from the cubesat industry, which means we may see multiple, smaller satellites working together as a constellation in the future.
Our guest today is Mallory Dodd, a Solutions Architect at iMerit, a company which specializes in producing labeled training data for clients in the early stages of kicking off their deep learning and machine learning projects. Dodd studied Anthropology at Texas State University, later on supplementing this with a GIS Certificate. She progressed through a few GIS roles before coming to her current one with iMerit, where her job is to educate new clients and to guide them through their experience with her company’s training data labeling services. What is Training Data? To back up here a bit, deep learning and machine learning are both types of artificial intelligence (AI). Ultimately, the goal of AI is to help humans solve problems more efficiently. Although they take a hefty time and resource investment to set up in the beginning, a deep learning or machine learning model can replace a variety of manual processes that will save man hours in the long run. If a model is trained well, it is not uncommon for the finished AI model to even surpass human accuracy in some situations.
Training data is a collection of annotated data (images, videos, vector data, etc.) that demonstrate to a deep learning or machine learning model’s algorithm how to repeatedly and consistently extract the information desired. Basically, training data is created by human annotators, and then fed into a model to “teach” it what to look for, and train it to mimic the human annotator’s decision process.
Training data can be very time consuming to build. The more complicated the objective of your model, the more training data it is going to take to train the model. Additionally, if you are hoping to pull multiple data points and add attributes (ie. this is a car + this is a car with 4 doors) from each training data source, this will also add time and complexity, but the payout in the end will potentially be better for what you need. Types of Annotation for Machine and Deep Learning Annotation is the human part of the process in generating training data for a model. Annotators will go in and physically mark out the features that they want the model to learn how to identify, and maybe add additional tags to help describe the image if needed. There are different types of annotation and labeling, and some are better matched to certain use cases than others.
The simplest form of annotation is simple classification of images. These are binary decisions about an image. Is it a dog or a cat? Is it day or night? Is the target present or not? This kind of training data can be generated quickly, but does not produce very detailed or informative outputs when compared to other annotation methods.
The coarsest method of marked annotation is the bounding box. Annotators will mark out the lower left and upper right bounds of the feature they are highlighting, and the generalized box represents the feature. It is even possible to create 3D bounding boxes, called cuboids, to use point clouds as training data. This method is great for uses like object tracking, as there are a lot of changes in movement going on that would be difficult to precisely mark, and an increased level of granularity would not add much to the output.
A step up from the bounding box is using polygons, which is basically digitizing the target feature. These allow the annotator to more precisely delineate the feature’s extent, and collect more specific information. For example, collecting polygons of cars, this could allow you to help train the model to more accurately identify if the car’s doors are opened, or closed. Of course, this method takes more time for the annotator than the bounding box, but the additional information and improved accuracy may be important for your use case. Other vector collections can be taken, such as points to track facial movements, or lines to track and predict routes, but polygons are the most common of the big three data types used here.
As far as raster training data methods, we have semantic segmentation, instance segmentation, and panoptic segmentation to choose from. Semantic segmentation is the practice of marking all of the pixels of the desired object as the “correct answer”. Instance segmentation is very similar, but adds the extra level of assigning a unique identity to each feature. For example, with semantic segmentation, we would have data tagged “car, car, car”, whereas with instance segmentation we would have “car 1, car 2, car 3”. Panoptic segmentation is the practice of each pixel being marked as something. Instead of just giving an identity to the target feature, we would also tag pixels of the background, sky, buildings, etc.
Raster training data classification methods are generally the most resource consuming to create, but they are known to produce the most accurate and precise products in the resulting model. Considering the potential payoff, this is definitely appealing, but there are some things to take into consideration if you want reliable and consistent results. Creating and Maintaining Quality Training Data Your algorithm will only ever be as good as your training data. This is why it is vital to understand what makes for high quality training data before getting deeper into the process.
The key element to getting expected results with your final model, is to use training data that is as close to the data the final model will be run with as possible. This is known as ground truthing. If you will be using the model with 3 band rasters and 1x1m cell sizes, your training data should be of the same type. Differences in resolution can cause issues in your model as it is now seeing the target in a different context than how it was trained too, resulting in loss of quality in the results, if it works at all. Some other differences to consider are the types of sensor used, the angle the target is viewed from, and lighting and weather conditions
Let’s say you have gone through the whole process of creating a machine learning model to identify cars in an image. You have annotated your training data, built and trained the model, and are getting the results you expected, but now your organization’s needs have changed and they need to know what color each car is as well. Do you need to start the model building process over from scratch? Thankfully, no, you don’t.
If you have a working model, you can choose to go through and update its training data to adjust the model to your new use case. In our car example, the annotator can go through and tag each of the training samples with just the color, as the car has already been delineated. Another option is to use the outputs from the previously created model to train your new one. If your model is giving you image chips of cars, then you are already most of the way there, and can simply classify the car colors using those outputs, and plug them into your next model.
A logical question is to wonder if the human role in this process can be removed. That is unlikely, as introducing human logic and thinking into the system is necessary in order to make sure the end product is meaningful to humans. When computers are left to their own devices they will take short cuts and make interpretations that make no sense to people, and can render the output useless. Keeping humans in the loop encourages transparency throughout development of the model, and ultimately results in a better product.
Artificial intelligence is, of course, still a young field. 10 years from now the landscape will have changed, and things we thought impossible we may have just failed to consider possible. As the technology finds its way into more markets, new use cases will develop, and with them, new ideas.
Our guest today is Ed Parsons, Google’s Geospatial Technologist of 15 years. He is also a Director to the Open Geospatial Consortium, and a frequent university lecturer on geospatial technologies. Parsons studied Geography at Kingston University, and got his MSc in applied remote sensing at Cranfield University. Starting out in academia, he transitioned to the private sector by working for the National Mapping Agency, as well as a few smaller roles before moving to Google, where he has stayed ever since. What is the Blue Dot on the Map? To put it simply, the blue dot is you. To put it less simply, the blue dot is your device’s predicted location based on the results of some unseen algorithms intertwined with the results of a set of fused location services. In the early days of web mapping services, the web maps were essentially just digitized printed maps, and they had pretty limited features when it came to positioning.
Everything changed, however, when computers got smaller and more mobile. They became so mobile, in fact, that they could move with us. The hardware and software to make individual location services happen had developed, and now the masses can, in the click of an icon, put themselves on the map. Mapping services such as Google Maps, Apple Maps, Waze, and countless others have made this utility ubiquitous to the point where we don’t have to think much about it. When we do think about it, most assume the blue dot is GPS. This is a common misconception, so let’s get into why.
An important piece of housekeeping, what people generally call GPS, is better-termed GNSS, a global navigation satellite system. It is the generic term for a group of satellites (a constellation) that send positioning and timing signals to earth. Every GPS is a GNSS, but not every GNSS is GPS. GPS is a US-centric system, with the official name being NAVSTAR. Other countries have their own GNSSs. Russia has GLONASS, China has BeiDou, and the EU has GALILEO. Fused Location Services A fused location service is a location service that builds a result based on combining spatial information from multiple sources. For mobile devices, there are three elements that generally contribute to the blue dot. 1. The Phone- Your cell phone is an advanced piece of technology which is capable of far more than just texting and Candy Crush. Phones need to be aware of their location to enable their most simple functionality, sending and receiving calls. A phone needs to be able to find the closest cell tower to itself and then the closest tower to the recipient in order to complete the connection. This coarse level of location information is a great start for narrowing down where exactly the device is. 2. WiFi Hotspots- The internet is nearly everywhere in this day and age, and this creates a geospatial advantage as WiFi hotspots have their own location information that can be broadcasted and used by other devices. Each router has a unique ID via the MAC (media access control) address. Their location information is built into massive databases which can be cached and queried based on the local geography of the mobile device in order to further narrow down its location. The caching element is especially helpful as it allows minimal network involvement, especially if you are generally staying in the same area. 3. GPS/GNSS- So, GPS is involved to a degree with fused location services. Considering that GPS is very taxing on machine resources, resulting in drastically reduced battery life, the goal is to minimize its role as much as possible. Furthermore, GPS needs a clear horizon to the responding satellites to produce the most accurate location, and this is not a common scenario for most use cases.
Generally, most of the grunt work for fused location services is being handled through WiFi hotspots. It would be impossible to deduce this just from scrolling around your favorite mapping app, as fused location service management is handled on the OS-level of your device. This is perfectly fine, as there is no reason to be manually involved in this process. The closest use case for this would be as an application developer. In development, there are some options that can be controlled to manage the level of precision needed for a location measurement in the app. How Accurate is My Location? Of course, accuracy and precision are two different things. As a quick review, accuracy is how close the result is to the true value, while precision is how close the results are to each other. Ideally, we want a result that is both precise and accurate. If you have spent enough time using navigation applications, you know that they work splendidly 99.9% of the time, but it is that 0.1% we are going to dig into a bit more.
We must remember that GPS is the most accurate and precise data source we can use to find our location, but battery consumption and the ideal hardware carry a high expense. This means location calculation is mostly handled by interpreting cell tower ranges, and WiFi hotspots. Both of these methods are less precise than GPS, but realistically, for most use cases this does not matter. If you are looking to pull a weather report for your neighborhood, your exact XY location to the nth degree is not needed, close enough is good enough.
Consider this, go to your favorite fast food chain’s website. There will likely be an option to Find a Location Near You. When you use this tool, you should be able to search based only on ZIP code, and you will get a list of usable results, relevant to your geography. Providing your exact address here would be overkill. Keep this in mind the next time you download an application that requests your location, chances are, it does not really need it.
Location information is highly private, and sensitive. Reputable vendors, such as Google, will never share your location information with third parties for exactly this reason. They also avoid storing any of this data long term for security (and data storage cost) reasons. Additionally, they take measures to separate your identity from your reported location, further reducing the risk of such information being used maliciously. Finding Locations in Cities Man made environments provide a special challenge for navigation. They are full of thick concrete, metals, and competing network resources, introducing opportunities for error.
If you have ever opened your mapping application, and it briefly shows your blue dot as being in some exotic location, such as Birmingham, chances are it has picked up a stale WiFi hotspot from that cached database we mentioned before. The algorithms driving the fused location services will pick up that one of these things is not like the other, and discard the red herring.
Another common issue is with subways and metros. If you are deep underground, zipping along in an electric train, there is a lot of interference going on with your mobile device’s location tools, assuming it gets any connection at all. Once you emerge above ground, your phone comes out of this darkness into the light, but has no background information to kickstart its wayfinding. It will scramble to give your blue dot its location, but it has the further challenge of determining your heading/direction. This is where you get people on the sidewalk spinning in cute circles trying to follow the erratic arrow trying to direct them to the nearest street corner.
Google has made strides in attacking this issue by integrating VPS- a visual positioning system. This takes advantage of your phone’s camera, as well as the vast database of spatial and structural information that Google has stored about the earth’s surface in order to pull key points from the environment to help orient you. They have even integrated augmented reality visuals to give you directions through your camera.