GIS: Recent Episodes
mapscaping.com
In this podcast episode, Jennings Anderson, a research scientist at Meta, discusses the Overture Maps Foundation, a downstream product of OpenStreetMap.
He explains his background in open map data and his interest in studying collaboration within the OpenStreetMap community.
Jennings then dives into the Daylight Distribution, an open data product produced by Meta, and how it combines building data sets from various sources into one unified theme.
Jennings emphasizes the importance of a stable ID system within the Overture Maps Foundation and the potential for easy conflation and integration of third-party data.
Jennings also explains the relationship between OpenStreetMap and Overture Maps, highlighting how they complement each other.
Relevant podcast episodes
OpenStreetMap Is A Community Of Communities
Cloud Native Geospatial
Cloud Optimized Point Clouds
The Rapid Editor
With regards to accessing Overture Map data, you might find this YouTube video helpful
https://youtu.be/fZj6kTwXN1U?feature=shared
Just in case you are interested in the Google building footprints here is a link to that :)
https://sites.research.google/open-buildings/
In this episode of the Mapscaping Podcast, we had the pleasure of speaking with Sam Hashemi, the CEO and co-founder of Felt, a browser-based mapping tool that is making waves in the geospatial community. Felt is not just another mapping tool; it's a platform that's revolutionizing the way we interact with geospatial data, thanks to its unique "upload anything" feature.
The "Upload Anything" Feature: Felt's "upload anything" button is a game-changer. It allows users to upload any data without specifying the type, and the system figures out the data on the backend. This feature sets Felt apart from traditional mapping tools, which often require users to specify the type of data they are uploading. The "upload anything" feature is a testament to Felt's mission to make mapping more accessible and user-friendly, reducing the technical barriers often associated with geospatial data handling.
Felt's Contribution to Open Source Projects: Felt is not just about creating a user-friendly mapping tool; it's also about giving back to the community. Felt is the first and only flagship sustaining member of the QGIS project. They are building an open-source tiling engine called Tippecanoe and also support Proto maps and the development of PM tiles. They contribute code to MapLibre and GDAL as well.
The Genesis of Felt: Sam Hashemi, a product designer by profession, started Felt after noticing the difficulty in creating a basic map, let alone a complex one. While there were powerful desktop software like QGIS and Esri, there was no modern, fun, and playful tool on the internet. This led to the creation of Felt, aiming to be the Google Sheets for maps.
Felt's Vision: Felt aims to make map creation fun, playful, and delightful. They believe in increasing the "GDP of maps," i.e., the number of maps created every year, by making the process more accessible and enjoyable. They envision a world where many more people are creating many more maps because it's fun, creative, and useful in their day-to-day lives.
Felt's Business Model: Starting in 2024, Felt will charge a monthly fee for using the software. However, there will always be a free tier for consumers. They take privacy seriously and do not sell or look at user data.
Conclusion: Felt is still in its early stages, having launched the company two years ago and the software one year ago. They are starting to see exciting ways people are using the software and believe they are at the beginning of something big. As Sam Hashemi puts it, "If I was to impress one thing upon your viewers, it's Felt. It's free, go to felt.com and you can sign up and try it. No need to listen to my words or think about it, go feel it in your hands, go get that Felt experience." So, why not give Felt a try and experience the future of geospatial mapping?
Rapid is a free open-source web-based editor for an OpenStreetMap. In the past the focus was on conflating AI-generated datasets with OpenStreetMap data but the future for this editor is conflating authoritative datasets with OpenStreetMap.
Humans are in the loop, people reviewing data authoritative datasets and adding them to OpenStreetMap with a few clicks!
So you might be wondering, what is Authoritative data? And perhaps it doesn’t even matter what authoritative means maybe the most important thing is it correct.
If you are interested in OpenStreetMap you might enjoy this episode
https://mapscaping.com/podcast/openstreetmap-is-a-community-of-communities/ which is a great introduction to OpenStreetMap as a project but also explains some of the commercial interest in updating the map which adds a lot of context to Rapid and its development and future.
Humanitarian OpenStreetMap Team
If you have not heard of Humanitarian OpenStreetMap Team this is well worth checking out!
https://www.hotosm.org/
Segment Anything
(SAM) can segment objects by simply clicking or interactively selecting points to include or exclude from the object. This makes it a user-friendly tool for image segmentation.
https://segment-anything.com/
Mapillary
Mapillary is a platform that provides street-level imagery and map data from all over the world. The platform is powered by collaboration and computer vision, which helps in generating and maintaining up-to-date, detailed maps.
https://mapscaping.com/podcast/scaling-map-data-generation-using-computer-vision/
GeoSpatial Jobs!
Drone Deploy’s Earthworks team is looking for an experienced Back End Engineer Full time / Remote
NV5 is looking for a Senior GIS Analyst
https://mapscaping.com/jobs/
Want to help? I could really use some support!
The promise of digital mapping is to provide a shared and real-time view of the state of the underlying system.
pg_eventserv is a free and open-source component that helps fulfill the promise of real-time event modeling and shared views in PostgreSQL.
By connecting to PostgreSQL and listening on specified channels, pg_eventserv captures database notifications and forwards them to web clients, enabling real-time updates and synchronization of data displayed on maps or other web interfaces.
pg_eventserv does one thing and one thing only: take events generated by the PostgreSQL NOTIFY command and passes the payload along to waiting WebSockets clients.
pg_eventserv is free and easy to install and you can find it here: https://github.com/CrunchyData/pg_eventserv
What this means is that any client can watch for notifications and update as changes in the database happen.
Real-time data!
Here is a link to a Youtube video demonstration of pg_eventserv in action!
https://youtu.be/UakRtYmoWow
I will let Paul Ramsey the creator of pg_eventserv explain all this in more detail in this episode.
If you want to reach out to Paul the best place to do that is http://blog.cleverelephant.ca/
Or if you want to listen to previous episodes with Paul you might find these interesting
Raster in the database?
https://mapscaping.com/podcast/rasters-in-a-database/
Dynamic Vector Tiles Straight From The Database
https://mapscaping.com/podcast/dynamic-vector-tiles-straight-from-the-database/
Spatial SQL- GIS Without The GIS
https://mapscaping.com/podcast/spatial-sql-gis-without-the-gis/
also ... If you are interested in spatial databases at scale ... you might find this episode interesting
https://mapscaping.com/podcast/distributing-geospatial-data/
pygeoapi is a Python server implementation of the OGC API suite of standards ... which might be really useful if you are thinking about upgrading from the first-generation OGC standards to the second-generation OGC standards
... or if need to implement a custom data source or custom functionality to your web services.
If you are using MapServer, GeoServer, MapProxy, QGIS server, or Deegree you might find this episode interesting!
Relevant previous episodes
Cloud-native Geospatial
Geoserver
Geonode
Sounds like a great idea right?
In this episode, Paul Ramsey explains why you shouldn't ... unless you want to ... and how you can ... if you have to.
You can find Paul's blog here: http://blog.cleverelephant.ca/about
Previous episodes with Paul
Spatial SQL
https://mapscaping.com/podcast/spatial-sql-gis-without-the-gis/
GDAL
https://mapscaping.com/podcast/gdal-geospatial-data-abstraction-library/
Dynamic Vector Tiles
https://mapscaping.com/podcast/dynamic-vector-tiles-straight-from-the-database/
Blog posts by Paul about Rasters in the Database
https://www.crunchydata.com/blog/postgres-raster-query-basics
https://www.crunchydata.com/blog/waiting-for-postgis-3.2-secure-cloud-raster-access
Check Out Our Geospatial Job Board!
https://mapscaping.com/jobs/
I am sure you have heard of ChatGPT by now so the hope of this episode is to give you some more context about what is it built on and how it works.
To do that I invited Daniel Whitneck back on the podcast
You can connect with Daniel here
https://datadan.io/
and listen to his previous episode here:
https://mapscaping.com/podcast/an-introduction-to-artificial-intelligence/
This is perhaps the quote for the episode that I have spent the most time thinking about
"We always thought AI would be logical and lack creativity - but it is almost the exact opposite"
This reframes the idea of being wrong to being creative which I think you could argue really depends on the context!
If you have not already played around with ChatGPT it's well worth spending the time to experiment with it ... while its still free ;)
https://chat.openai.com/auth/login
Further listening
If you have not already listened to this episode about computer vision and GeoAI you might find it interesting. Listen out for the discussion around plausible / realistic data and real measurements - I think this gives more context to the use cases for generative AI
https://mapscaping.com/podcast/computer-vision-and-geoai/
You might also enjoy this episode about fake satellite imagery
https://mapscaping.com/podcast/fake-satellite-imagery/
BTW I have started a job board for geospatial people
feel free to check it out!
Data is what data does - more about that later on ;)
This episode focuses on designing for privacy, how do we create value from location data without sacrificing personal privacy?
Well, you might start by adhering to the Enhanced Standards For Precise Location Information which means that information about sensitive places like churches, hospitals, military bases, and LGBTQ+ spaces isn't misused or sold. Plus, they protect our exact location from being shared with law enforcement or bounty hunters!
Yes, that's right bounty hunters!
You might also think about adding noise to the data, maybe you want to blur the time stamp and look at everything in aggregate?
It turns out the location data is not just classified as personal data but is actually classified as sensitive personal data in privacy law so if you are going to work with it you really need to understand the risks involved.
One of the really interesting ideas mentioned by Elizabeth Hein VP of Compliance & Data Protection was the idea of regulating Use, Harm, and Risk instead of sensitive data
Data Is What Data Does: Regulating Use, Harm, and Risk Instead of Sensitive Data
If you want to learn more about POI data and why points of interest data are so hard
check out the episode called All of the Places in the World: https://mapscaping.com/podcast/all-of-the-places-in-the-world/
On a side note, I am working on a side project, it's a job board for geospatial people, and you can find it here https://mapscaping.com/jobs/ it's still in the development phase but feel free to check it out!
Foursquare's ongoing work to map every place in the world
About The Guest Kyle Fowler is the senior director of engineering at Foursquare. Before joining Foursquare in 2011, he was a third party user of Foursquare developer tools. Ever since, he has worked on different projects, in varying roles to solve a variety of location problems.
What Is Foursquare? Foursquare is a cloud-based location technology platform that supports building solutions based on a deep understanding of location. For the general public, Foursquare is a location-based social network that facilitates meeting up with friends and discovering new places. Users can check-in to places and show their network where they are, and what they are doing. On the other hand, commercial companies and organizations use Foursquare APIs and advertising enablement products to gain insights from the physical world.
Foursquare Swarm Swarm is a life-logging app that keeps a user’s history of interaction with the world. It is a perfect diary for keeping a personal log of the places you have visited, alongside photos, and information about the friends you have been with, and the experiences you had. Logging places into Swarm can happen passively in the background or through active interaction with the app. In active logging, you have to interact with the app in every place you visit. But passive loggers can also actively build their history by periodically reviewing their history feed to confirm the places they have visited.
How to Download Your Swarm Data One way to download your Swarm data is by requesting it through your profile. The data will include all your check-ins and any other information you logged. The other way is through the Foursquare API after signing up to be a Foursquare developer (anyone can sign up). Using an API is the best way to pull your data if you want to use it for running analysis or building your own features.
How Is Foursquare Mapping All Of The Places In The World? Crowdsourcing is a huge contributor to Foursquare location data. But Foursquare may not have users in every corner of the world. To get a good amount of coverage the data is obtained from several other sources that include machine-generated approaches and from other companies. For completeness of coverage, Foursquare purchases licenses for regional data generated by other trusted companies that have a particular interest in a specific area.
How Does Foursquare Control the Quality of Location Data? All the places submitted at Foursquare go through a quality audit process to ensure that the location is real. The initial screening removes low accuracy points that may introduce skewed locations.
Machine learning models are also used to review the input sources and assign a confidence score of whether a place exists. Further controls also involve sending samples of places to human moderators for review.
Foursquare understands that the negative outcomes of being wrong about a place are worse than the place not being listed at all. If people using that location information actually go there and find that it does not exist, it results to a bad user experience and a negative perception on the app.
A quality auditing of the data helps to take out all the bad sources and ensures the recorded locations are an accurate reflection of the real world.
Maximizing Accuracy with the Geosummarizer Foursquare’s Geosummarizer is a model that analyzes the inputs for a certain POI (Point of Interest) and selects the right coordinates.
The Geosummarizer helps to solve different challenges such as when people record slightly different coordinates for the same place they check into. This is more likely to happen in dense urban environments especially when the user is inside a building.
To try and find the right coordinate, the Geosummarizer compares the input coordinate to the geocodes of that place’s address and infer whether the coordinate should exist within that place. The process then picks the best coordinate that has the most corroborating features with that address.
If there are multiple clusters of coordinates for a given location, the Geo-summarizer tries to assign them appropriately to different geocodes. A place can have multiple geocodes; and the users will use the one appropriate for them based on the use case.
For instance, an app may want a pickup and drop-off location of a POI (Point of Interest). The Geosummarizer also outlines various levels of venue hierarchy for describing relationships of places within places such as a gate inside an airport terminal.
Locations for mobile things like food trucks are treated separately from other categories. The summarization process for generating a new coordinate for that venue takes the latest data into account much more heavily than it would for a brick-and-mortar store that is not expected to move.
Mobile POIs can get updated in real-time – on every single check-in – as long as the coordinates are believed to be legitimate.
Foursquare’s Taxonomy of Places Foursquare has over 1200 categories for classifying places across the world. In an effort to make the places locally familiar, some of those categories are only visible in certain countries. For instance, one would not expect a Chinese Food category in China, or Italian restaurants to show up in Italy.
Foursquare tries to adapt the categories system to reflect the depth of reflectiveness for a particular country. Categories are re-evaluated on a monthly basis to ensure they are showing up appropriately, and whether new categories should be added.
Mapping all the places in the world and making them locally familiar is a challenging task. However, Foursquare’s knowledge of the world is continually increasing. As more locations are added and their model is continuously improving, Foursquare is steadily moving towards achieving this goal.
es I am going to publish in partnership with Foursquare and the idea is to use it as a reference for later episodes about Privacy and location data, Knowledge Graphs, AI, Location Based Marketing and Big geospatial Data in the Browser.
Protomaps is a serverless system for planet-scale maps, it's an umbrella project consisting of a few different components one of which is PMtiles.
PMtiles is “Cloud Optimise Geotiff” for web mapping, what this means is that you can build a base map and host it without the need for a server!
PMtiles is a single file that you can access via HTTP range requests in the same way that you can access data within a Cloud Optimised Geotiff with the important difference that PMtiles can also contain vector data!
What this means is that you can create your own base map, and host it on something like Amazon S3 object storage at a fraction of the cost of other base map solutions!
During this episode, you will hear Brandon, the founder, and creator of Protomaps, talk about scarcity, and well I have never really thought about base maps as being a scarce resource I can definitely see how a product like PMtiles could remove some of the barriers to entry for a lot of creativity in terms of base maps.
More information on Protomaps is here: https://protomaps.com/
Tippecanoe
https://github.com/felt/tippecanoe.git
https://bertt.wordpress.com/2023/01/06/creating-vector-pmtiles-with-tippecanoe/
Relevant podcast episodes
Cloud Optimized Point Clouds
https://mapscaping.com/podcast/cloud-optimized-point-clouds/
Cloud Native Geospatial
https://mapscaping.com/podcast/cloud-native-geospatial/
Microsoft’s Planetary computer
https://mapscaping.com/podcast/the-planetary-computer/
Stamen Design - Full Stack Cartography
https://mapscaping.com/podcast/full-stack-cartography/
If you have any questions or comments, let me know, I would love to hear from you!
Modeling The Vision Of A Better City With LIDAR
LIDAR is a fast and efficient method to capture high-resolution data with accurate measurements. But in this episode, the focus is not on using LIDAR to make better measurements. Rather the talk is about how point cloud data can be used to tell a story. The guest tells how creating a film with 3D LIDAR data of a city provided a medium that could be consumed by the public to understand how a city could be transformed.
About The Guest Benjamin Muller is a team leader at Leica Geosystems in the R&D section. Doing mechatronics, they combine disciplines such as software development, mechanics, and electronics to develop devices such as drive chains for laser scanners and total stations.
Benjamin was part of a project that used LIDAR data to create a film that communicated the vision of a better city. He emphasizes the importance of using easily understandable films to communicate complex details. In his words:
“Facts do not always speak for themselves. If a visualization can be wrapped around the truth, and packaged in a story, it can reach a lot more people. And this could be a driver for change.”
How Is LIDAR Useful In 3D City Modelling? Getting insights into a whole city is an important element in the process of transforming a city into a future-proof city. It is not possible to get this kind of insight when walking through a city because at that moment you are only viewing the city from a very small perspective.
LIDAR makes it possible to make 3D models of the whole city, with accurate measurements that preserve the integrity of the city features. Visualizing a city in this way provides a wider perspective on how various changes would impact a city. Viewing a city and its surroundings in a new perspective unlocks new ideas and better strategies of transforming a city. Additionally, LIDAR data collection can be done whenever required to capture any new changes that have happened in the city.
Drawbacks in Working with LIDAR During LIDAR data collection, massive amounts of point clouds are captured. Even for a small city, the size of data captured can easily grow into several terabytes. With data of this size, many LIDAR experts have reported that a great part of their work, over 80%, is filtering the data. It can take months to process LIDAR data for a city.
Processing LIDAR data for a city is costly both in terms of time and money – which can run into several hundreds of thousands of dollars. Using AI in processing can considerably reduce processing times, but even then working with the data requires expensive and powerful equipment. For most people, who may just need a one-time visualization of a part of a city, it is uneconomical to buy such expensive infrastructure just to visualize a small part of the city.
How LIDAR Was Used To Visualize a Better and More Sustainable City A project undertaken in the city of St Gallen in Switzerland used LIDAR data to create 3D models of the city. It added contributions from different experts to create subsequent versions of how the city can be transformed. These versions were pieced together to create a 30-minute long film that helped to communicate to people a vision of a greener, more sustainable city.
The advantage of using a film as a communication tool is that it is easier to understand. It effectively communicates the message to people who may not deliberately read detailed books on how to transform a city. Additionally, a film can efficiently reach a wider audience in society and inspire people to become a part of the change for a better future city.
From A Vision to A Strategy The film of the city was created to communicate a vision of a better future city. But it also transformed to a strategizing tool. With this visualization, decision-makers could formulate a step-by-step strategy that can be applied to make the city more sustainable and healthy for both the people and the whole environment. This led to the creation of 16 books alongside the film, which contained detailed information on how to do the desired transformations in the city.
Other Uses of 3D City Models Various sectors including tourism can benefit from 3D models of a city. Using augmented reality the 3D models can be used to support digital tourism. While visiting a site, tourists can also view the site from different perspectives, and experience things that would otherwise not be obviously visible.
For cities where you need to report the changes you are doing to structures on your property to city authorities, visualizing the changes in a 3D model provides an opportunity for understanding the impact of the changes to the city and the environment.
How Can We Democratize 3D City Data? With AI it is possible to build services that process LIDAR data on the cloud. These services can then be accessed through a browser by the public. In this way, more people can visualize the data without having to buy very expensive infrastructure or become an expert in visualization. Providing easier and cheaper ways for the public to consume LIDAR data is an important step in its democratization. But if only a few specialists are able to use the data, it will still remain a very expensive luxury to have it.
HxDR (Hexagon Digital Reality) HxDR is one example of a publicly available platform for viewing 3D models of cities. You have to register first before you can view detailed 3D city models. At the moment, there are models of several cities in Europe and the US on the platform. Some of the cities have high-precision geospatial data and you can also purchase parts of the city that you are interested in.
A platform like this is only the beginning of a transformation to have access to 3D data, but we expect there is more coming in the future.
If you are interested in more technical episodes about point clouds you might enjoy these! The Point Data Abstraction Library Cloud Optimized Point Clouds Bathymetric Lidar Lidar from Drones Lidar from Space
Warning! this podcast episode is not as boring as it sounds!
While geospatial standards are boring on purpose ... this episode is not .-
If you woke up this morning wanting to listen to a boring podcast episode about geospatial standards this is not for you!
Scott Simmons ( OGC’s Chief Standards Officer )
https://www.ogc.org/about/team/scott-simmons/
helps us understand what a Geopose is and how it might be used, why we need GeoRSS, and something called SensorThings!
You will also learn about the PubSub standard for the syndication of spatial data and why streaming data is not always the answer.
We also discover what the most boring geospatial standard is and when the shapefile "might" die.
All in all, this is a light-hearted look at a very serious topic and I hope you enjoy it!
What to know more about the Open Geospatial Consortium? this is a good place to start! https://mapscaping.com/podcast/the-open-geospatial-consortium/
If 80% of all data has a spatial component why do we need to approach mature and emerging markets differently? With everyone racing to democratize location technology why is GIS still a valuable skill? What are consultancies looking for when they hire geospatial consultants?Answers to these questions and much more in this episode where we focus on geospatial consulting, as a business and career path. Todd Slind - VP of Technology at Locana.coConnect with Todd on Twitter or LinkedIn Recommend Podcast Episodes The Business of Web MapsBuilding a web-based mapping tool into a businessA Business Built on Open-Source GISBeing A Graduate Geospatial ConsultantHiring and Being Hired for Geospatial JobsGetting Where You Want To Go In Your Geospatial Career
COPC – A Cloud Efficient Data FormatIn this episode, the discussion revolves around cloud-optimized point clouds. Our guest is Martin Dobias, the CTO at Lutra Consulting. Coming from a background in computer science, and a passion for geospatial, Martin has been part of a team that has done a ton of interesting work in the open source geospatial world. Today, he shares about their latest, state of the art developments in working with point clouds on the web; Cloud Optimized Point Clouds (COPC).
What Does Cloud Optimized Point Clouds Even Mean?Point clouds are sets of individual points plotted in 3D space. They are typically very large datasets, as they must capture a real space in great detail. For instance, point clouds for a whole country may easily be several trillions of points, and many terabytes of data. Handling these large datasets on the web requires a lot of bandwidth to download and process.
The files for Cloud Optimized Point Clouds are structured with indexes for each part of the dataset. The index structure makes it possible to stream only the parts of the data that are required, without having to download the entire dataset.
What is Point Cloud Indexing?Point cloud indexing structures a point cloud file, making it possible to find any particular point of interest in the file without having to scan through the entire dataset. COPC files are internally indexed using a 3D structure of cubes called Oak trees. One part of the file is the data itself, and the other part contains the hierarchical information of where to find each cube.
At the root level of the hierarchy is a single cube, which at the next level is split into eight smaller cubes. The splitting continues subsequently up to the highest hierarchical level. As the cubes get smaller, they contain a smaller amount of data, which saves bandwidth if a user is only interested in a small part of the data. It is similar to traditional tiling, but with the added 3D context.
How COPC Files Are Accessed Using HTTP Range RequestsRange requesting is a feature of the HTTP protocol used to access information more efficiently from web servers. Instead of a server sending an entire file to a browser, the range request feature allows the browser (client) to define a specific part of the data that the user is interested in. Subsequently, only the requested part is sent by the server.
For cloud-optimized point clouds, the server will simply go through the hierarchy and find the cube or multiple cubes that satisfy the request, and return these. This process is much faster than having to search through the entire dataset. Moreover, only the point cloud cubes that satisfy the browser request are sent by the server, which reduces the bandwidth used.
Converting LAS Files to COPCLAS is a standard open source format for point cloud data interchange. However, it is less efficient to work with on the cloud since the data in it is not indexed. This means the whole LAS file must be loaded before it can be queried, sidestepping the efficiency we see with COPC indexing.
Conversion of LAS to COPC can be done in QGIS using Entwine. Practically, when a point cloud file is loaded to QGIS, it is automatically converted to COPC. QGIS structures the data automatically in order to make operations more efficient as opposed to working with unorganized datasets.
When point cloud files are converted to COPC in QGIS, the new COPC file contains all the information in the original dataset. Unlike other software that may discard some information when processing a file, for COPC nothing is discarded. This makes the COPC format great not only for visualization purposes, but for analysis as well.
What Infrastructure is Required to Serve COPC Data?Serving cloud-optimized point clouds does not require any special infrastructure between the server and the client. It is easy to host and get the data to the client without complex infrastructure, i.e. there is no need for something like GeoServer, MapServer, or QGIS. Just having the COPC data in blob storage somewhere is all the infrastructure that may be needed.
Compatibility of COPC FormatA compressed LAS file is called a LAZ file. COPC is much like a LAZ file. This means that applications that accept these formats will also be able to work with COPC files without having to implement special support. The only difference is that they will not be able to use the extra features of internal indexing in the COPC file.
Where Can You View COPC data?
QGIS offers support for viewing COPC – both stored locally on your device, or remotely in the cloud. Using a link that points to a server containing COPC data, QGIS will load the data on demand according to the queried range. The data is further cached in QGIS, which makes subsequent data loads and views much faster.
There are also a couple of projects coming to life that explicitly support cloud-optimized point clouds. An example is the web viewer built by Hobu. With the link to a COPC data server, the web viewer will fetch the relevant COPC files and render them in the browser.
The PDAL LibraryPDAL (Point Data Abstraction Library) is a library that contains a set of tools for working with point cloud data. In the QGIS environment, many users are familiar with PDAL’s feature for the data access of point clouds. Many of the library’s other functionalities are unknown to a lot of users. It contains a dozen features to classify, filter, export, convert point cloud to raster or meshes, amongst others.
The main reason why many functionalities in PDAL are not popular among ordinary users is due to the complexity in using it. The PDAL library uses pipelines that need to be crafted manually when working with point cloud data. While this may work well for advanced users, ordinary users find it a bit too complicated.
After a successful crowdfunding campaign, Lutra Consulting and several other partners are working to reduce this complexity, and make the functionalities in PDAL more user friendly. The project seeks to build a simple integrated toolbox within QGIS for point cloud data processing. The same way there are integrated toolboxes in QGIS for working with vector or raster data, there will also be one for working with point cloud data. These developments may be expected across the next two QGIS releases; in February and June of 2023.
What is the STAC Protocol?STAC (SpatioTemporal Asset Catalog) is a protocol for easy access to spatial and temporal data. It makes it easier to index, discover, and work with geospatial information. STAC is commonly used with satellite imagery but recently it is increasingly being used for distribution of point clouds as well.
How Were Point Clouds Streamed Before COPC?No doubt, before COPC there were some existing formats for streaming point cloud data. In the open source world, one of them is the EPT format, built by Hobu Inc.. The EPT format closely compares to raster tiles; but for point clouds. It is structured in a big directory with individual files (tiles). COPC files have an advantage over EPT format, as opposed to having thousands or even millions of files in a folder structure, COPC is just a single file – which is much easier to work with. In the proprietary world there are a couple of formats as well, one being the I3S format from ESRI, which supports point cloud data and other 3D data. There is no doubt we will continue to see explosive growth supporting point cloud data management. Stay tuned with the MapScaping Podcast to make sure you stay current on the latest and greatest developments!
PDAL - Point Data Abstraction Library
Full Stack Cartography – Think Like a Designer for Better MapsThis episode features Alan McConchie, a leading cartographer at Stamen. The discussion centers on mapping and visualization; and explores the usefulness of adopting a design-first thinking approach to the map making process. Alan has over 8 years experience making maps that are as beautiful as they are useful. Together, we will gain a deeper insight into full stack cartography, and how we can leverage it in our own work.
Spatial Data Visualization vs. Non-Spatial Data VisualizationAlthough they are both ultimately design tasks, there are some distinct differences between spatial and non-spatial data visualization. For one, there is lesser liberty in spatial visualizations than there is in non-spatial data visualization. Unlike in the latter where the designer can take liberty in choosing where to place objects and what shapes to use, spatial visualizations usually have to maintain location integrity. If the locations in a map are distorted, the map will be misleading and fail to serve its purpose. The flexibility in maps is mainly found in the choice of the symbology, color ramps, and fonts used. Even these areas are often guided by standard cartography rules, so it's important to make the most of the creative space provided if you want your maps to stand out from the rest.
What is Full Stack Cartography?Full stack cartography is seeing the world as a challenge of design, and how it can be effectively mapped for a particular use case, or to communicate your topic of interest. To back it up a bit, full stack is the idea of taking your entire workflow, and all of the areas of your system that it touches into consideration together. Full stack cartography incorporates design thinking into the map making process right from the beginning, throughout the process, and until the end. Writing the code and scripts used to process the map data is heavily guided by the use case of the end product.
Ideally, every decision, at each stage of the process should contribute towards making the end product the best it can be for its use case. All the decisions made when making a map are important; from how the data is stored in a database, to how it is processed, to how the end product is displayed. The ripple effect of all these decisions throughout the process can make achieving the end goal easier (if done right), where ignoring them can make the process more difficult, inefficient, and altogether frustrating.
Why Are Base Maps Pre-Generated? A base map is a pre-generated background map that provides context to other map layers that are overlaid on top of it. They generally contain generalized land uses, streets, water features, and building footprints, amongst other landmarks. Pre-generating base maps makes loading web maps faster, and more efficient, by leveraging imagery tiles. Since creating base maps is a complicated process that usually requires a huge amount of data, generating them on the go every time is computationally intensive and slow for devices with low to standard computing power – which is the case for the majority of users.
In order to improve the user experience even further, base maps are divided into many small pieces, called map tiles, so that a browser does not have to load the entire base map, but rather only loads the pieces (tiles) of the area that a viewer is interested in. This is faster and more efficient than loading the full dataset.
How Do Map Tiles Work?Using map tiles is a popular and common technique for displaying maps on the web. After generating a map, it is divided into many small pieces (usually square shaped, be sure to look into Hex Tiles); which, when joined, form a seamless single map.
Tiling maps improves the user experience by reducing the size of the dataset that a browser has to load when displaying a map. Less data to load means displaying a map is faster, and uses less bandwidth. This is significant as web maps are already data and display intensive due to the many additional features and layers that need to be loaded alongside the base map, especially if those layers contain a lot of attribute data. To view an area of a map, the browser will only load the tiles that cover that area. When a user pans the map, more tiles that cover that area are returned. Previously loaded tiles may be cached so returning to your previous extent will be faster than if the map was loaded fresh.
How Are Map Tiles Created?After creating a map, tiles can be generated using map tiling tools or services such as Mapbox Tiling Service. Tiles can be in a vector or raster format. The tiles are generated for different zoom levels to allow users to zoom in or out of a map as they may need. At higher zoom levels, the map tiles are further split into smaller equal tiles that display more details. Each zoom level has its pre-generated tiles that are easily loaded to a browser upon request. Most programs, like ArcGIS, have a default tiling scheme and thus create tiles for specific zoom levels that are most optimized for their viewers. If, however, you want your map to be drawn at different intervals, you can create and publish a custom tiling scheme.
Geometric SimplificationCreating map tiles includes deciding which details will be included in the tiles, and which will be filtered out. In order to improve efficiency even further, cartographers try to reduce the size of data stored in each tile as much as possible. Some simplification is applied in the lower zoom levels to exclude details that may not be visible or useful in that zoom level. For instance, an application to show bike trails in a park may only display the trail lines at lower zoom levels, but then include the attributes like the trail names, or topographic lines at higher zoom levels. A cartographer decides which details to include at each zoom level and cuts out what is not required to avoid storing unnecessary data in the tiles. Having light weight tiles makes them even more efficient, fast, and useful for many mobile applications.
Styling Map TilesAfter deciding what map details appear at each zoom level, it’s time to decide how they should look. The tiles only contain the shapes and attributes of the map features. In order to apply styles such as colors and textures, a stylesheet is used. The stylesheet defines how the data stored in the tiles will be displayed. Design tools such as Mapbox Studio, or the open source option Maputnik, can change the color, line thickness, and other styling characteristics of a feature and display them instantaneously for review.
What is a “Good Map” in Cartography?A good map may mean different things in different use cases, and to different people. Generally, a good map is one that finds the right balance between beauty and usefulness. No doubt we all love a beautiful map, but if it is too beautiful then users spend more time admiring it than actually using it. This is not a good map as it does not successfully and efficiently convey its message. Keeping the distractions to a minimum is one important trait of a good map.
Design-first thinking helps to create good maps that are useful, as well as visually appealing. Everything from the choice of colors, to fonts, to avoiding emphasis on less important features will contribute to whether a map is successful in its application. The ultimate measure of a good map is how well the map serves its intent. This of course includes making it visually appealing so that people are willing and able to use it in the first place.
Is Cartography Changing?In the world of cartography, a lot has changed already. The tools used to make maps decades ago are not the ones being used today. The way maps are displayed today is different from how they were displayed in the past as well. The way maps are used have evolved too, and this is why the approach to cartography has also evolved over the years.
Today, maps are not only created for humans but for machines as well. For instance, self-driving cars need maps for navigation, and have to process a lot of simultaneous inputs.
In the future, more changes can be expected as further advancements are realized. Technology is getting better by day and soon we may see even more new products in cartography. For one, since personalization is a big thing these days, we may start seeing more personalized maps. This will come hand in hand with the loss of privacy that seems commonplace, and even expected nowadays. The developments seen in indoor mapping suggest we will soon see seamless outdoor-to-indoor navigation.
There is much we can expect from the future. In order to avoid being surprised, get involved now.
Recommended Podcast Episodes about Cartography
Grow your career — or reimagine it entirely, what do you do if find yourself wanting to make a change halfway through your career? This episode with Thierry Gregorius from https://truehorizoncoaching.com/ is more of a discussion than an interview and if you are thinking about making a change in your career it might help you answer the question "what do I do next"
Recommended Podcast episodes focusing on careers in Geospatial
Getting Where you want to go in your geospatial career
https://mapscaping.com/podcast/getting-where-you-want-to-go-in-your-geospatial-career/
Geospatial Side Hustles
https://mapscaping.com/podcast/geospatial-side-hustles/
I quit my job
https://mapscaping.com/podcast/i-quit-my-job/
Mentorship, leadership and career advice
https://mapscaping.com/podcast/mentorship-leadership-and-career-advice/
Business ideas for geospatial people
https://mapscaping.com/podcast/business-ideas-for-geospatial-people/
What is your idea
https://mapscaping.com/podcast/what-is-your-idea/
Python Maps
https://mapscaping.com/podcast/python-maps/
For more episodes just visit https://mapscaping.com/podcasts/
and sort all the episodes by "geospatial career"
Consider supporting this podcast on Patreon
https://www.patreon.com/MapScaping?
Or go to MapScaping.com to find out about sponsoring our website
reach out on Twitter https://twitter.com/MapScaping
or LinkedIn https://www.linkedin.com/in/danielodonohue/
This is a story about a peer-to-peer mapping technology that is enabling people to "fight maps with maps"
https://www.digital-democracy.org/
You can find Mapeo here
https://www.digital-democracy.org/mapeo/
Promoting OSM projects
OpenCage and Mapscaping are working together to help projects based on OpenStreetMap reach a wider audience.
Projects will be selected to be featured on upcoming Mapscaping podcast episodes, with all costs covered by OpenCage!
Apply here!
https://forms.reform.app/NAvQ41/opencage-mapscaping/af9MFV
Recommended Podcast episodes about Off-line mapping
MerginMaps - QGIS offline and in the feild
https://mapscaping.com/podcast/qgis-offline-and-in-the-field/
https://mapscaping.com/podcast/two-mobile-data-collection-apps-you-need-to-know-about/
More podcast episodes on GIS and GIS careers can be found on our website https://mapscaping.com/podcasts/
Consider supporting this podcast on Patreon
https://www.patreon.com/MapScaping?
Or go to MapScaping.com to find out about sponsoring our website
reach out on Twitter https://twitter.com/MapScaping
or LinkedIn https://www.linkedin.com/in/danielodonohue/
After 12 years of working as a GIS specialist / Consultant, I quit and I am ready to move on.
Looking back at the last 12 years if I could go back in time and give myself some advice I would tell myself these 8 things.
- Boring Work won’t necessarily lead to interesting work
- Getting good at something typical leads to doing more of that thing
- Have more empathy for the professionals that you work with
- Think more about where you are trying to go
- Network
- Find a mentor
- Sometimes it's just not a good fit
- You don’t get to decide what is broken
Things I need help with
- Share the podcast with a friend, a community anyone that you think might find it interesting or helpful
- I am looking for projects based on or around OpenStreetMap - this is part of a promotional campaign that I am doing together with OpenCage - More information here
More podcast episodes on GIS and Geospatial topics can be found on our website https://mapscaping.com/podcasts/
Consider supporting this podcast on Patreon
https://www.patreon.com/MapScaping?
Or go to MapScaping.com to find out about sponsoring our website
reach out on Twitter https://twitter.com/MapScaping
or LinkedIn https://www.linkedin.com/in/danielodonohue/
The problem of unification: Spatial data comes in many different sizes, shapes, and formats making it a difficult and time-consuming process to join data for visualization, exploration, and analysis.
Enter the Hex Tile system!
Contact Foursquare: connect.foursquare.com/mapscaping
Check out Unfolded https://foursquare.com/products/unfolded/ https://www.unfolded.ai/
Introducing Hex Tiles: https://foursquare.com/article/introducing-hex-tiles-our-next-gen-tiling-system/
Example Maps
https://studio.unfolded.ai/public/e86787a1-4871-4fbb-8eb8-33445e81ca73
https://studio.unfolded.ai/public/e3a6d2df-1784-4686-972b-9d3793c2515d
https://studio.unfolded.ai/public/beb3cf0d-211d-4e5c-b08d-663da9a0631d
Recommended Podcast Episodes
Dynamic Vector Tiles Straight From The Database
This is a story about a hobby project called Scribble Maps that grew into business… but it's also a story about the opportunities and challenges of creating geospatial tools for non-geospatial professionals.
Jonathan Wagner
“Its easy to get people to make a map today, but how do you get them to come back and make a map tomorrow?”
Special Offer for Listeners for the MapScaping Audience
ScibbleMaps.com/mapscaping
More podcast episodes on GIS and GIS careers can be found on our website https://mapscaping.com/podcasts/ Consider supporting this podcast on Patreon https://www.patreon.com/MapScaping?
Or go to MapScaping.com to find out about sponsoring our website
reach out on Twitter https://twitter.com/MapScaping
or LinkedIn https://www.linkedin.com/in/danielodonohue/
Building a Business from an Open Source Project: A Case of White Box Tools The guest on the show is John Lindsay, the creator of Whitebox Tools and a professor of Geomatics at the University of Guelph. John introduced us to Whitebox Tools in a previous episode and described it as a collection of open source geospatial analysis tools that can be embedded in other applications. In this episode, John is sharing with us how they are building ways to generate revenue from Whitebox Tools, while maintaining its status as an open source project - yes, this is possible! When is a Business the Right Solution for an Open Source Project? Being a developer for an open source project could be an experience of two extremes. On one hand, while it may be immensely rewarding to encounter day to day users of the software you have developed, but the experience could be quite stressful as well, especially as the user community expands.
Usually, open source developers have their day job, and only spare their personal time to work on the open source project. When the user community grows, the demands of this larger community may take a toll on the time of the developer. It may be a daunting and potentially draining task to have to respond to too many requests from users. This could lead to developer burnout, which could lead to a project dying out.
A growing user community is a good thing, but for the developer this growing demand can be taxing. At some point, it may be difficult for the developer to provide a satisfactory level of support to the user community as it scales. This means that the project would need more hands to support its userbase.
Generating revenue for an open source project builds more stability for the project, and means that it might be there in the future, with the right support and resources.
If you are having doubts about the idea of building a business from an open source project, then think of the 1000s of users of open source software who not only find them useful for education and research, but also in generating revenue for companies. If there are companies that can use open source tools to generate revenue, then it makes sense that the developer of the project should be able to leverage it in generating revenue as well. The Process of Building a Business from an Open Source Project Building a business from an open source project is not very different from how enterprises build their businesses. The only may be the challenge of preserving the integrity of the project as strictly open source. Outside of this, the business world presents a lot of the same challenges. Finding a Co-Founder Doing business alone could be really lonely, especially for a developer who is already overwhelmed by the demands of the project and community. It may be useful to find a co-founder who can help with the business side of things. At this point, it is important to know what your limitations are, and use them as a guide for the things you should be looking for in a co-founder. The right partnership is one in which the strengths of the partners fill in on the weaknesses of the other and contributes to achieving the goals of the company. Laying Out the Business Foundation In exploring the ways to generate revenue for the project, there are several things that should are imperative for the project’s success: maintaining its integrity and defining its success Keeping the Integrity of Your Goals A growing user community may be a good indication that the original goals of the project are serving the users well. When developing a business model for the project, it is critical to prioritize maintaining the integrity of those goals as much as possible, especially in the regards of it being an open source project. If the user community feels short changed, the project users may shrink and the possibility of the business being a success will diminish. It is very important to uphold the goals that keep the project intact, even when setting new goals. Defining Your Own Success For businesses built from open source projects, success may not mean the same thing as those built in commercial enterprise. Founders can have their own definition of what success is for the business. Sometimes this could mean the ability to sustain a workforce that supports the user community, and keeps the project alive.
Whitebox Tools are exploring ways to generate revenue that will be useful in keeping the project alive in the long term with adequate user support, learning materials, and improving and developing new tools. To them, success is being able to maintain and develop the Whitebox toolsets, and support the community that has gathered around it. This has helped Whitebox keep its focus on the user community, and maintain much of its original framework of the project. Ways to Earn Revenue from an Open Source Project One way to generate revenue from an open source project is by creating new paid tools. These paid tools could be those that extend the functionality of the free tools as in the case of Whitebox Tools Extension. In this way, the status of the free tools would still be maintained as open source as revenue is generated from the paid extension. In order for this to be successful, the paid tools should be compelling. Creating Compelling Products A compelling product is one that users are interested in. It may be a challenging task to know what tools the people want, or whether they will be willing to pay for them. There are some things that can help guide the creation of paid tools. One is looking at various thematic areas that your tools are geared towards and figuring out some tools that would have a great interest in those application areas. This will give an idea of where to focus development efforts. If you can create a toolset that fixes all the problems a certain type of user has, you become essential to that user.
Engaging with existing users of your product can also be a great resource for knowing where to focus development efforts. There is much that can be extracted from the conversations where users describe how they use a product, and how they would like to use it.
Gaps in a particular field can also provide an opportunity for developing a paid tool. A good place to look is in areas of recent advancements. Spotting a gap will enable you to create a solution that will be used by people working in a particular application area, and its more likely you’ll beat someone else to it. The “Pay as Much as You Can” Pricing Model “Pay as much as you can” is a pricing model that lets people donate to free tools. The Whitebox Tools website has this pricing model in the form of buttons displaying amounts between $0 and $100 on their tools download page. This presents an opportunity for users to support the project, even if they may not be able to afford the paid tools. It may also help to put in a note explaining how the donations will be used in support of the project. Steering the Business towards Success Thinking of the right tools to create, and having the right pricing model is not all that there is to succeed in this business. Increasing Product Visibility In marketing, the ideas that spread are the ones that win. It’s hard to achieve success if the project you are working on is invisible. It is unexpected for people to care about things they do not know exist or that they cannot find. This is an especially big challenge for products that are predominantly backend.
Usually, it is the frontends that get more attention than the backends that power them. Opting to create your own frontend may be a way of making your project more visible, but if your solution is designed to power other applications, this could be the wrong use of time. Alternatively, entering strategic partnerships with essential popular frontends may be useful in growing your appeal.
Cultivating various types of non-financial support may also help increase user awareness. For instance, reaching out to long term users and asking them if they could support you by mentioning your business on their website, blog, or social medias. You could also simply ask for permission to showcased them as users of your tools on your business website. Effective Communication Communicating effectively is a key part of the success of any business. Although you may create a really wonderful tool that provides an innovative solution to a problem you know people have, if it is not communicated properly, it still may not be successful. Effective communication means using the right words and images, or packaging them in a other way that helps people to understand what the tools can do for them. Try different communication styles gracefully, while acknowledging that many businesses find it more difficult to communicate a tool to users than creating a tool itself. Commitment and Confidence Commitment is super important. If you put something out in the world and it fails, have the confidence to try again. Repackage it, find other creative ways to communicate it and try once more. If you are using avenues like Twitter, LinkedIn, or even a blog or newsletters, it will help to think that you're speaking to different, if not overlapping groups of people in each of these avenues. It is important to reach out as inclusively as possible. The Website A website can be quite central in telling the story about a business. It is the first place where people from other avenues and social media will learn about you. A lot of effort should be put into the website to make sure it effectively communicates what the business offers. Imagine that people know absolutely nothing about your business and be as specific as possible about all the capabilities offered. How a User Community Kill, or Keep an Open Source Project Alive? Without doubt, a developer has a certain responsibility that comes with creating a software and making it available for people to use. After all, a project lives and dies by its user community. Its users, whether in an educational, government, or commercial setting, have a certain responsibility to ensure that the project thrives. If the developer is responsible for everything, it can result in burnout and risk the continuity of the project in the long term. Thriving open source projects are the ones which gain the support user community. Building a Business Is an Iterative Process One thing to always have in mind when building a business is that success does not usually happen the first time. If it does not turn out well the first time, try it again with a different approach and an open mind. Iterate and improve upon your process and eventually it gets better. There may be many struggles along the way but part of the journey is learning from mistakes.
WhiteboxTools – A Toolset for Geospatial Analysis
Our guest on the show today is John Lindsay, the creator of Whitebox Tools, a toolset which contains over 500 geospatial analysis tools. John is a professor of Geomatics who got into geospatial in the early 2000s. Many of the tools in Whitebox have novel functionality that are not found in any other software and are free to use.
An Overview of WhiteboxTools
Whitebox Tools is a package for geospatial analysis that can be embedded into other software in order to facilitate building other types of applications. The journey of creating WhiteboxTools began with the development of a Terrain Analysis System (TAS). This was a full-blown desktop GIS that was fairly widely used at the time of its creation. Following further developments and tools additions, Whitebox GAT (Geospatial Analysis Tools) was created, from which WhiteboxTools emerged.
WhiteboxTools evolved from Whitebox GAT after a shift in focus from the frontend to the backend, with the aim of creating a portable toolset that could easily be embedded in other software applications.
This includes desktop GIS like QGIS and ArcGIS; as well as scripting languages like Python, R and NEM, which enable users to integrate any tool in Whitebox into their larger workflows involving other geospatial packages. Whitebox is also embedded in other geospatial applications, such as the open LIDAR toolbox, geemap, and Leafmap, and many other applications that harness the analytical power of Whitebox Tools in their analyses.
WhiteboxTools Is Open
White box is an open source software with open access. Here, the notion of open access means to remove barriers for the users of the software so that they may examine, inspect, and understand the source code itself. Users can view the source code of any of the open core tools in Whitebox, using any frontend that has the functionality to view code.
Users can also port the toolset and translate the tools into another programming language that they are more comfortable with.
Whitebox Tools Is a Standalone Software
No prerequisites are required to be able to use WhiteboxTools. Whitebox is designed to have zero dependencies. It is a standalone executable that users can download and use in their platforms.
How Many Tools Are in Whitebox Tools?
WhiteboxTools is comprised of two categories of tools: The WhiteboxTools Open Core, and the Whitebox Extensions that extend the functionality of the open core tools.
There are about 465 tools in the WhiteboxTools Open Core, and approximately 63 tools in the Whitebox Extensions. All these tools have been developed in the RUST language. RUST is a low-level programming language that was picked by Whitebox with the aim of making the tools as fast as possible, while using the least amount of memory possible.
What Can You Use Whitebox For?
Whitebox contains a broad set of tools that are useful across a variety of applications. Generally, these tools are for use in solving spatial problems. Some of the tools are intended specifically for certain applications, for example, LiDAR Toolbox contains a collection of tools which can take a typical LIDAR dataset from raw point clouds all the way to the end product needed, such as an interpolated digital elevation model (DEM).
Some geospatial applications that make use of WhiteboxTools include: Wetland mapping projects, Geological resource inventorying, Digital archaeology, Soil mapping applications, Landslide modeling, Geomorphological modeling, Forestry, Spatial hydrology, and Modelling.
Where Do the Tools in Whitebox Come From?
Most of the tools in Whitebox come from research and experience. Whenever a cutting-edge tool for a certain type of process is developed during research, Whitebox becomes the platform to disseminate the information about that innovative tool. Some tools are also developed from user requests.
If there are multiple users requesting the same feature, then Whitebox will make it a priority. At the same time, if it's just a single request, but it's a really interesting spatial problem that had not been thought about before, a unique solution can be built then it may result in a tool as well.
A number of tools are also a result of helping people solve problems they encountered. Whenever a person has a geospatial problem and emails Whitebox to see if there could be something that can be done about it, if a tool is built that solves their problem, it is published on Whitebox in the open core where everyone else can use.
This is one of the reasons why there are a number of very niche-specific tools in Whitebox, which solve a very specific need. They are usually inspired by a very specific need, by a certain user.
Environments to Use WhiteboxTools
WhiteboxTools can be used in desktops as well as high computing environments like supercomputers, web-based environments, and even mobile environments. Although it is currently not being used in mobile environments, there is no reason it couldn’t be.
At the moment, mobile GIS is still much more focused on front end data visualisation for people in the field than it is for raw data analysis, but we could easily see this change in the future.
How Should WhiteboxTools Be Used?
Users may use WhiteboxTools in whatever capacity they can envision. If a tool is intended to perform a certain task in a particular application, but its capabilities may be applied in a creative way in very different scenario, then users are encouraged to explore that capability.
Whitebox fully supports the idea of users using their capacity to apply WhiteboxTools in different ways, as evidenced by their responsiveness to user input.
Turning Science into Practice
Whitebox is a platform that communicates usable implementations of cutting-edge geospatial solutions to geospatial practitioners. In many areas of research, the ideas and solutions developed are usually published in journals within academia.
While this is fairly accessible, many practitioners may still not have access to it, or may have access to it but not to the software artifacts that implement the solutions. WhiteboxTools bridges this gap and brings science (ideas and solutions) into practice by making usable implementations accessible to the whole community, rather than just the academic community.
Does Whitebox Accept Tool Contributions from Users?
The concept of Whitebox was developed with the goal of allowing user contributions to the toolset. One way is by suggesting ways to improve and optimise the existing tools.
This is why the source code is open access. The other way is by developing a new tool and presenting it to be published in WhiteboxTools. The new tool should be developed in RUST language, and will of course be subject to review.
Who Is WhiteboxTools Designed For?
In many fields today, people are leaning towards niche solutions. If something is for everyone, then it is also for no one since it does not differentiate itself in the market.
WhiteboxTools brings a new twist to this. It has a broad set of tools cutting across different applications, but what stands out is that these tools are very niche specific. Many of these tools are developed to perform a certain task in a particular application – novel tools that provide unique solutions that cannot be found elsewhere. Whitebox tools are not niche in terms of their applications but in terms of their solution.
Recommended Podcast episodes
Mergin Maps – Offline Data Collection for QGIS Projects
The guest for this episode is Peter Petrik, the CPO of Lutra Consulting. He previously worked in the automotive industry before transitioning to geospatial about eight years ago. Ever since joining Lutra Consulting, he has made use of his programming knowledge and skills, while collaborating with others to develop products and contributions to the open geospatial community. One of their recent developments is Mergin Maps, an open-source software that allows users to take QGIS to the field.
What Is Mergin Maps?
Mergin Maps is an integration that allows users to collect geodata using their mobile devices or tablets. Teams working together can use Mergin Maps to synchronize their data seamlessly to the cloud, or a server. The QGIS plugin enables users to easily access their data in the cloud, and utilize QGIS to post-process or analyse their project.
How Mergin Maps Works
In order to use Mergin Maps to conduct field surveys for a QGIS project, the first step is installing the Mergin Maps Plugin for QGIS. The plugin will ‘package’ your whole project into a folder, and make all of its layers available for use offline. If the project uses an online map, QGIS processing tools can be used to generate its layers for offline use. The plugin also runs several validation checks to ensure that the tool will work correctly on a mobile device. Once everything is set up, a single click on the synchronise button will take the project to the cloud. The project can then be downloaded to a mobile device, and used offline within the Mergin Maps Input app.
Every time a user clicks on the sync button in the app, any new changes are pushed to the cloud, and a new version of the project containing those changes is created in the cloud. It is easy to track the changes made to the project and identify the user that made them.
Mergin Maps also has an automatic merging feature. Users working offline on the same project, will have their data automatically merged by Mergin Maps once they sync it to the cloud. This removes the need to do it manually, and can save a lot of time if there are many users in the field. Mergin Maps does not have restrictions on the number of users that can be added to a project.
Data Storage When Using Mergin Maps
Mergin Maps is open source, so users are not restricted to using just its cloud. Users may opt to deploy their own servers to store their data.
The Mergin Maps Cloud is an option for users who do not wish to have the weight of server infrastructure and maintenance on their shoulders. There are various Mergin Maps subscriptions for users depending on their storage needs. For academia and personal use, Mergin Maps is free to use for up to 100 MB of storage.
Mergin Maps has features that help users reduce the size of their data, and cut down their storage needs. For example, there is a QGIS tool where users can set the resolution of all the photos captured in the i.e. 1 MB via the app. This helps to avoid storing high resolution photos whem they are not necessary.
Mergin Maps User Management
Mergin Maps allows a project owner to manage the users that can participate in the project. A user can be added to a Mergin Maps project as a reader, writer, or administrator.
Readers can only access the project and view the data, while writers can create new versions of the project, either in the mobile application, or from QGIS. Administrators can invite other users, as well as change user permissions.
Each user will need to create a Mergin Maps account before they can be added to a project. Once they have an account, they can use the same log in credentials to sign into the mobile application and, download the project from the cloud onto their device. They can then start working in the project (i.e. digitise, edit data, syncing etc.)
What Expertise Do You Need to Use Mergin Maps?
For the Mergin Maps QGIS plugin, some knowledge of GIS is required to be able to set up a project and push it to the cloud, and later access the data and do analyses and generate reports.
No GIS knowledge is required for users of the Mergin Maps Input app. The developers acknowledge that the users of the app are likely from different fields, with varying levels of knowledge. Therefore, they kept the design simple and easy to use. On average, a new user can learn how to use the Mergin Maps Input app in about 15 minutes!
Mergin Maps is flexible and has plenty of users from various industries and applications including agriculture, utilities and fibre optic companies, consulting firms, and many more.
Mergin Maps Integration with External GPS Tools
If the location from a mobile device is not very accurate, a user can connect to an external GPS. Mergin Maps supports most of the common external GPS tools which a user can plug in or simply connect to. They will then be able to use the external GPS’s precision for their data collection. Only Android is supported for this functionality at this time.
Controlling Data Inputs on Mergin Maps
Mergin Maps offers a number of widgets that can be used to control the values accepted by the Mergin Maps Input app.
For instance, if an input field is set to only accept integer values, an integer widget will appear on the data collector’s mobile device as a slider, or text box where they can only input a number.
Further limits can be set to define whether the number must be positive, or may not exceed a certain figure. For dates, a widget will pop up on the mobile device as a calendar. These functionalities are great for helping to manage data quality and data assurance standards.
There are also widgets for lists, checkboxes, and other common widgets and constraints. Users can also use the QGIS Expression Engine to write calculations for derived fields.
There are lots of possibilities supported on both the QGIS plugin, and Mergin Maps Input app, where users can apply some basic quality assurance methodologies, and help with capturing quality data in the desired format. After data has been pushed to the cloud, users can make use of the Python integration to write scripts in QGIS, or on their server to further validate the data from an input field.
Mergin Maps Community Edition
Mergin Maps are advocates for open-source GIS and that reflects in their products. Their users are not vendor locked within Mergin Maps infrastructure. If a user is tech-savvy enough, they can set up Mergin Maps, and use it on their own. There is a public GitHub repository with a Docker compose file that users can run for themselves for their personal projects. In case they encounter problems, they can join the Slack chat and seek help.
Mergin Maps Enterprise Edition
The Enterprise edition is best suited for companies that need to hold data in their servers due to legal or security reasons, or internal rules. Mergin Maps facilitates this by offering their knowledge from their years of experience running cloud services, to help in the deployment and maintenance of each company’s server, making sure everything works smoothly and efficiently.
Recommended Podcast Episodes
Joining us today on the podcast is Gregor Willhauck, the Geospatial Cloud Strategist at Trimble, to talk about digital twins and how they integrate with geospatial. Gregor’s background and degree is in forestry, but he has spent most of his career in product management positions, working largely with satellite imagery, GNSS systems, and now, digital twins and modeling. Focused on the big picture, he is always looking for the best ways to use the cloud to further enable digital twins and geospatial tech.
What Are Digital Twins?
With the rise of commercial and recreational drones and LiDAR scanners, digital modeling has become much more commonplace in GIS and other industries. This has unlocked all kinds of cool and useful applications of models and point clouds for both everyday, and scientific uses. Standard digital models are awesome, but digital twins have entered the space in response to a unique set of needs and spatial questions.
So, what exactly is a digital twin, how is it different from a regular model? Well, a digital twin is a virtual representation and digital counterpart of a physical asset, which is updated in real time. The key defining element to a digital twin is this real time aspect. A standard 3D model is captured once, and represents a single frame in time, where digital twins are the result of regular scans, and fuel and end to end data pipeline.
Defining what qualifies as “real time” can be a bit tricky, and largely is going to rely on what you are monitoring with the scans.
An active construction site for example may need to be modeled several times daily, even hourly, to realistically capture a crew’s progress. A completed construction project, however, may only need to be scanned daily to monitor just enough change to understand when maintenance is needed.
Both of these qualify as “real time”, but one is a bit more real than the other.
Digital twins are data dense, and can require a hefty amount of resources to create and maintain. This is why they pair so well with the cloud. Cloud computing and storage allows these models to be created at a frequency that matches the source image collection, resulting in minimal lag between when the UAV is on site, and when the finished product is on screen. This adds legitimacy to the final product, and allows decisions based on these products to be made with a higher level of confidence.
How Are Digital Twins Made?
The most straightforward answer here is that a digital twin can be created with pretty much any of the sensors and vehicles already used for remote sensing. Again, the wheel is not being reinvented here, it just has more RPMs.
Some of the most common data sources for fueling digital twins are drones, handheld scanners, and even robots.
These vehicles will generally carry cameras, or LiDAR scanners, and perform a flight plan for mapping, just as if they were creating a one time model.
The resulting images may be directly streamed to the cloud, where they will undergo processing before being uploaded to whatever end application is needed for the workflow.
Robots are exciting here because they can allow the entire digital twin workflow to be automated. After an initial human-led flight to build a map of the area, which is used to plot the robot’s course, the robot can then be scheduled to complete, and upload the collection to its overlords around the clock.
Photogrammetry is the primary tool for processing that creates these models.
Photogrammetry is when software identifies “tie points”, or common points between two or more images, used to “stitch” the images together into a continuous product.
Some clever geometry related to changes in where the sensor is in space, and the Pythagorean theorem, allow 3D reconstructions, and highly accurate remote measurements.
Digital twins can be supported by GNSS/GPS devices as well, especially to track movement within the context of the larger environment. GNSS devices are the easiest platform to monitor and mimic in “real time” because the data is produced quickly, and is easy to collect and transfer. Different data sources will have different levels of latency, and a digital twin can be composed of multiple data sources.
Uses and the Future of Digital Twins
Digital twins are a very cool, futuristic technology, but with all the required investment, what can be expected in return, and is it worth it? When it comes down to it, digital twins are a logistics tool. They allow an unprecedented level of monitoring, and enable highly valuable insights through visualization, and analysis of the vast data they are capable of producing.
A real-time geospatial data model itself is not the product, the information it contains is.
End users want to be able to extract features, like structural damage, or measure change, like on an active construction site.
Being able to do this remotely results in some financial and time savings associated with traveling, and allows stakeholders to catch issues as they occur, rather than at the next scheduled manual inspection.
Another major use of digital twins is measuring traffic or “flow” throughout an area. For example, if a digital twin is built of an office complex, then internet of things (IoT) devices like smart elevators, or even sliding doors, can paint a picture of congestion, or inactivity in certain areas at various times. Similarly, this technology can be used for fleet management to track where and when various vehicles are moving through space.
Digital twins of course serve as an excellent visualization tool. They are eye-catching and engaging, and can be accessed from nearly anywhere.
Virtual reality and mixed reality views can fully immerse a user in the space, opening up the option for interactive engagement, making markups, and viewing alternative setups or configurations in real time.
All of these activities ultimately serve to help promote collaboration and efficiency in an organization. Everyone is able to access the same information as someone on site, which we may see continue to grow in importance with the advent of work from home. As computing, storage, and UAV resources become more affordable and accessible, we will likely see new unprecedented uses for digital twins continue to emerge in both real, and digital life.
The Spatial internet of things - Sensor Up
Big Query
Google Earth Engine
The Planetary Computer
Augmented Reality
Joining the podcast today is Eric Jensen, currently a data scientist with Climate Engine. After obtaining his BA in Geography, he completed a masters in Ecology adding context to his research with remote sensing through the University of Montana. Eric has gained a variety of experiences during his career, ranging from working with web tools, to conservation and research work, to a data analysis focus, and even to creating public-facing geospatial educational media. He is going to share his experience and expertise with approaching the geospatial job market, and finding what you’re looking for. Making a Job Transition Making a career move can be scary, whether it is within your current company, or trying somewhere, or something new. The first step in making a transition is deciding if now is the right time, and if not now, then what would the right time look like? What can you do to move up that timeline? Here are some things to consider.
Finding your dream job starts with identifying what your dream job is, and who is hiring for it.
A company’s focus, industry, and culture have a huge influence on what the day to day of that role may look like, and if it’s a good fit.
This includes considering who your coworkers and clients might be, and if you would be compatible. If your preference is for the research and innovation associated with academia, you might like a company that employs a lot of scientists. If you like ambitious colleagues with a range of experiences, a startup might be a good choice.
Identifying the specific job you want is going to rely on your experience, interest, and the perceived opportunities.
GIS is often described as a tool, and many people find greater success using GIS as a secondary skill to support whatever their expertise in their main industry is.
If you are not sure exactly what niche to climb into, start asking yourself some deep questions about where you could see yourself going, or better yet, have someone else ask you these questions and garner their feedback. Spend some time reflecting in order to identify exactly what your next move looks like.
An essential, yet intimidating element of changing jobs is discussing that move with your boss or supervisor. Some bosses aim for stability, and may not react as well to the idea of a change, but some supervisors might surprise you and end up being an excellent resource in your job search.
A boss that can act as a reference is immensely valuable, as they can provide your next boss with specific examples of your skills and what you bring to the table. If you have a boss that is supportive of your growth, start that conversation with them, it might be easier than you think. Tools for Success in Finding a Geospatial Job Once you have decided it is time to make a change, it’s time to get actionable. Entering an iterative process of research and self-reflection can help you to uncover some truths about where your strengths, weaknesses, and interests lie.
The most productive tools for learning more about career growth in GIS are informational interviews.
An informational interview is an informal conversation with someone you find to be a role model in your industry.
These low-stakes conversations allow you to source honest answers on questions you have about nearly anything. You may discuss questions surrounding salary, curating a social media presence, or get some insight on how they approach tough interviews.
Preparing for an informational interview first requires finding someone to interview. You might not have to look far, however, as your existing network might already hold some great options.
As a general rule, most people enjoy talking about themselves, so do not be afraid to reach out to request interviews.
In order to prepare questions, try checking out your interviewee’s LinkedIn, or cruise through their company’s website. Use the list of questions you build to help guide the conversation. As a natural part of the conversation, you may find a lot of these questions reflected back at you. It is okay to not have answers for all of them, especially if you are still relatively young in your career. Take this opportunity to notice gaps in your knowledge, both in industry topics, and in knowledge of your own professional priorities and interests.
After concluding an informational interview, reflect on what you have discussed.
Did you get what you came for? Distill the main takeaways from your conversation into a few sentences and write them down somewhere for later to jog your memory if needed.
Another great resource for building insight on how people in GIS approach their careers is by listening to podcasts like this one! The MapScaping Podcast hosts a curated archive of years of conversations with industry professionals, and we even allow you to filter for Career Focused episodes. GIS podcasts like MapScaping, Seeing From Above, Minds Behind Maps, and Geodorable can provide a lot of the same value as conducting your own informational interview, but without all the effort. How and Where to Look For a GIS Job After identifying your niche in GIS, start identifying companies that need it filled. You might start by looking within your existing network, or by following companies you have interest in on LinkedIn or Twitter.
Plan to take a look at GIS job boards which may allow you to get even more niche.
ClimateBase, for example, posts jobs specifically surrounding the climate industry. Similarly, if you are interested in a company, follow their public job board postings to be notified as soon as new positions become available.
Before applying for your potential new job, spend some time refreshing your portfolio, resume, and CV. HR softwares are notorious for filtering out applications based on the presence of keywords, so take the extra time to tailor your resume to the job description.
Another popular addition to a CV or resume is including a link to your portfolio website (use a PDF, never a Word doc!).
When a decision-maker pulls up your site, you gain immensely valuable ‘facetime’ with them, and a chance to impress with your well-curated portfolio.
Of course, in business, not everything is entirely based on merit. While it is certainly possible to get a job through the traditional application process, anytime you can expand on a personal connection you have with the hiring company it will help your chances of making it to that final interview.
An added benefit of job-seeking within your network is it gives you a chance to gain some background and insight on your potential future company.
You can ask questions about the culture and growth opportunities that might not feel as natural when you reach the formal interviews.
GIS is an ever-expanding industry, revealing potential new applications through an iterative process of innovation. By investing some time into reflecting on how you fit into the larger schema of the field, you can be more confident that you are making the best choice for your future as you pick your next move.
Our guest today is Dr. Adam Symington, the lead data scientist at Geollect. Although now he is known for his impressive geovisualization, built using Python, his professional training is as a computational chemist. After completing his schooling with University of Bath he stuck around to complete postdoc work on the development of machine learning models used to track the movement of atoms within materials. At Geollect, he uses a similarly inspired approach to track the movement of ships using AIS data. Python plays a significant part in this work, and is the driving technology behind his spatial visualization hobby project- PythonMaps. The Path to Learning Python Anytime someone new to geospatial asks the greater GIS community what the best skill they can learn to increase their value is, the answer is the same- Python. Highly praised as flexible and easy to learn, Python is an excellent choice for those looking to bolster their resume. Of course, just because a group of computer scientists say Python is easy to learn, does not mean that the process is necessarily intuitive to those learning it as their first programming language.
Everyone has their preferred style of learning. There are a huge variety of books, tutorials, and full-on web courses designed to teach Python to beginners, many of these resources are even free.
Regardless of which medium you choose, it is important to start from the beginning, and dedicate some time to really understanding the fundamentals of Python.
Internalizing explicit explanations of what is a variable, what is a function, what is an object, etc… lays the groundwork for the more complex concepts ahead, such as defining methods, loops, and arrays.
Once you are comfortable with your knowledge of the basics, it’s time to start coding. The most important part of your Python journey is to stay excited about what's next. Find a problem that you find super interesting, and then find as many ways as possible to solve it using Python. The motivation to answer your original question will be essential when it is time to power through the inevitable knowledge gaps and challenges you will encounter in your projects, and when solving real world problems. What is PythonMaps? PythonMaps began as COVID lockdown project, where Dr. Symington was experimenting with creating Python-driven geo visualizations to further his own skills. As time went on, and he developed more expertise, and consequently higher and higher quality maps, he went public. Now you can find his work on Twitter, LinkedIn, and Reddit.
PythonMaps is the perfect example of why it is valuable to learn Python through personal projects. When there is genuine interest driving a project, it is much easier to find the motivation to learn new technologies and skills to support it.
Although Python offers thousands of libraries, only four are really used for most PythonMaps projects. These are matplotlib, geopandas, rasterio, and rioxarray.
Matplotlib and geopandas are classic data visualization libraries. Matplotlib’s focus is more on traditional datasets, but it does have a number of mapping functions. You can also check out the Matplotlib Basemap Toolkit API to add additional functionality and context to your creation. Geopandas is a geospatial expansion on the highly popular pandas data science library. It provides functionality to read and write geospatial data into dataframes, as well as a slew of great options for geospatial data analysis, summation, and visualization.
Where the above options are generally geared towards vector data, fear not, there are programmatic options for working with raster data in Python as well. Two useful libraries are rasterio, and rioxarray. Both of these libraries allow the reading and writing of a variety of popular raster formats, rioxarray has the added benefit of explicitly supporting cloud optimized geotiffs (COGs). These libraries can also provide core functionalities like reprojecting, resampling, and masking rasters.
There are many more data vizualization libraries available, but it is worth keeping in mind that for beginners, the most popular libraries will be the easiest to work with.
This is due to the larger knowledge bank of documentation, and existing troubleshooting conversations on platforms like StackExchange.
Having a mission in mind and playing around with these libraries is a great way to get started on producing your own content, but if you need some specific ideas, or a bit more help, Dr.Symington has a number of tutorials here. What Are the Benefits of Mapping with Python? In today’s information age, the power of a strong social media presence cannot be underestimated. This does not mean you need to be producing daily, or even weekly content, but it does mean that you should be investing some conscious time and effort into what does go public. Curating a professional online presence pays its own dividends, but even if you don’t get famous, your time will be far from wasted.
If your goal is fame, great. As you spend some time building up your Python and cartography skills, start thinking about how you want to present your creations, and yourself. Do you want to create a brand, like MapScaping, or would you rather take the personal route and present yourself as your brand, like Joe Morrison?
Building a brand allows you to distance yourself from criticism, and grow the project beyond what you alone can accomplish. On the other hand, making your brand personal may make it easier to attract a following.
It also has the potential to amplify the successes, and failures, that you encounter as there is no buffer between you, your work, and the criticisms of the internet.
The internet is infamous for harsh criticisms. Behind the safety of a screen, people are more willing to share their unfiltered opinions, but this is not always a bad thing. If you stay open-minded (and have a thick skin), those criticisms can be read as feedback.
Paying attention to negative comments that have some substance to them can show you where you still need to improve, in the same way that praise can tell you what you are doing right.
If your objective is not fame, then what does the return on investment look like here? Well, as many say, it is about the journey, not the destination. Learning Python and improving your cartography skills can do great things for your career and portfolio. It shows that you can succeed at self-directed learning, and can create value as an independent worker. Being able to communicate effectively is one of the most valuable skill sets in data science, and maps are a classic tool for visual communication.
Remember to focus your projects on topics you enjoy, and no matter the outcome, it will never be time wasted.
mentorship leadership and career adviceWelcome to another episode of the MapScaping Podcast. My name is Daniel, and this is a podcast for the geospatial community. If you're a long time listener of this show, you might recognize that delightful Australian accent that you just heard as Denise McKenzie. She's been on the show before, and she is the community and ethics partner at PLACE. Before she got there, she had an incredible career in geospatial, and that's what we're going to be talking about today.
We're going to be talking about mentorship. We're going to be talking about leadership, and, hopefully, offering some career advice and career encouragement for people that are still trying to find their way in geospatial.
Welcome back to the podcast, Denise, it's a pleasure to have you here again, and I'm really looking forward to this conversation. You've had this really broad career in geospatial, and that's what I'd like to focus on today. Before we dive into that, for the sake of context, would you mind letting the listeners know who you are, and how you got involved in geospatial?
Denise McKenzie:
Thanks for the invitation to come back, Daniel. It's always nice to be on the podcast. For those that don't know me, I'm Denise McKenzie. I really fell into geospatial about 25 years ago after having done an undergrad in international politics and public policy. It was really through a quite amazing individual called Bruce Thompson down in the Victorian government who took a good 45 minutes to wow me, so to speak, with the power and the capabilities of geospatial, and using maps for good in the world. I stayed working there doing innovation and policy work for about 12 years, really focused on that nexus between public and private partnerships and what can be achieved there. From there, I moved to OGC, the Open Geospatial Consortium international standards body, and worked as the head of outreach and communication there for about six and a half years. Following that, I became a consultant and worked with the Ordnance Survey and the media network for about two years through what's called the Benchmark Initiative that was really looking at the ethical use of geospatial and location data throughout the world. That resulted in me becoming one of the co-authors of a thing called the Locus Charter. It's a 10-principle charter looking at how we ethically and responsibly use location data. After all that, I joined a new not-for-profit organization called PLACE. There I'm their community and ethics partner. I look at how we build out a new global membership organization that's really focused on ethical and responsible use of data, as well as using data for good, particularly in developing nations.
Daniel:
Wow. You've had this really broad career so far, and I'm sure there's a lot more to come. I would like to talk about mentorship. I believe you mentioned someone called Bruce earlier on, and I know he was pivotal in your career, so I'd like to touch on that. First, I'm really curious to know, do you consider yourself to be a technical person?
Denise:
I’m probably less technical these days, although people constantly remind me whenever I say that, that I can usually engage quite well in a lot of technical discussions. Back in my early career in government, I was definitely deep in the world of GIS and creation of maps. I'd say these days, I'm more technical from the perspective of that architecture and strategy, but really looking at how new innovations and technology fit into our existing stack of geospatial tools. That's my level now more than being a day-to-day map making practitioner. To be fair, I did a master's in sustainability last year and went back and actually did a module in GIS. I've been getting back into ArcGIS Pro in the last 12 to 18 months, and it was quite fun to go back and make some maps.
Daniel:
Do you think you could be in that position to be in architecture, to be looking at different infrastructures and how they can fit together without cutting out room for knowledge of GIS and geospatial?
Denise:
I think so in that, you can come at it from different angles of the data world. You have to have some level of grounding and understanding of how data works. I know that the knowledge that I've got, and certainly one of the amazing things about that first role I had in government back in Australia, was that I was on a policy team, but I was on a policy team that sat next to people that were making maps on a day-to-day basis. We weren't sitting in our own little world doing policy. We sat with people who were day-to-day building, collecting data, working with users, etc.. I know that a lot of my technical knowledge comes out of watching that day-to-day experience and the challenges of data collection, of not getting the data you want, of having to clean it, and trying to work out how to share it. The level of knowledge I've got is definitely a learned on-the-job level of knowledge, more than in an academic context.
Daniel:
The reason I'm asking you these questions is because I really want to highlight that it is possible to have an extremely successful career in this space without being a deep technical person, or a programmer and machine learning specialist, and that it is possible to create a future for yourself. I want to get back to the earliest stages of your career, because you are involved in a mentorship program. I believe it's through the Women in Geospatial community. You, yourself, had a mentor when you started out earlier on in your career, Bruce Thompson. Would you mind telling us why this person was a great mentor? What did they do?
Denise:
I've had a lot of great mentors across my career. It's one of the things that I would say is so important to anybody. It’s not so much about progression, but I think about how you develop your understanding of what you want out of your career and what you want to do. Bruce, in particular was my first boss. He was the guy that really brought me into the world of geospatial. Outside of that element of Bruce, as a leader, he was really one of these people who quite intrinsically understood the value of diversity in teams. He had a leadership style where he was about supporting the nurturing of the teams, the nurturing of his staff, and helping them to understand and foster collaboration, and the value it brings.
He was always a really great mentor because he was looking to use his position of leadership as a mechanism to encourage people to be the best they possibly can be. He'd be the advocate. He'd always have your back. For me, as a mentor, I got to watch a leader that, ultimately, completely influenced the way I operate as a leader. All of my aspects of working with teams, of working with partners, and building memberships around the world were heavily influenced by this guy. I probably characterize him as what I would call a servant leader. He really looked for what he could do for all of these people that he was responsible for. How could he make them shine? How could he give them opportunities to improve and to develop the skills and interest areas that were relevant and best for them in what they're doing?
He was quite the amazing human. Sadly, we lost him to cancer late last year, which was quite a shocking thing for the geospatial community. He's greatly missed, because as well as being a brilliant mentor, he was an incredible visionary for where and how geospatial can be used. Going back to your comment about the career of geospatial, he actually started out in the world of architecture, and was an architect before coming into and becoming a leader in the geospatial realm. You're very right in saying that you can absolutely have high, and highly valuable, careers in geospatial coming from a huge, vast, different array of areas.
Diversity in GeospatialDaniel:
I think one of the first comments you made about Bruce was his ability, or passion, for building diverse teams. What does diversity mean for you when you think about building a team?
Denise:
For me with diversity I always imagine that what you're trying to do is build a team that somehow tries to capture the broad range of diversity that there is in the world. The global population is amazingly diverse, and different, and comes from a whole variety of different perspectives. Part of it is about trying to address that ethical and responsible use. If you build a team with a very similar set of people, from very similar backgrounds, you are only ever going to build a solution that contains very similar experiences from those people. If you build a team that is hugely diverse in age, cultural and religious backgrounds, and interest and skill areas, you're going to get people bringing in huge swaths of different experiences.
It might be that they have a background working with deaf people, or blind people, or vulnerable populations, people with all manners of different life experiences. Consistently over my career, what I've seen is where you have diverse teams, you get far better solutions that meet a far greater realm of the population and what they need.
Daniel:
It's one thing to be diverse, in terms of gender, or ethnicity, or beliefs. It's a whole other thing to let that diversity shine out, to be brave enough to be yourself. Now that you are working as a mentor, how do you do that for people? How do you encourage them to accept that diversity is a feature and not a bug? I think the tendency is to want to fit in and to be the same as everyone else.
Denise:
One of the things that I always say is that every single person in the world is unique. You bring your own background, values, experiences to the table. If I look at how successful my career has been, it's been because I've recognized the things about me that are different. Rather than feeling like that difference is a negative thing, I've said that's what makes me special. That's what makes me different. When I'm talking to my mentees, I try to encourage them to have the confidence to understand that you are special, just because you are you, with your life experiences, and with your people that you know and your network.
You are unique, you are valuable, and what you've got to contribute is going to be valuable too. That could be because you are from a different generation, or because of the country background that you've got. It will always be valuable to the teams that you are working with. It's an element of understanding your self-worth and trying to quiet that imposter syndrome component that sometimes rattles around in your head. When you look around the room and you think, I don't look like these people, or I don't talk like these people- instead of seeing that as a negative and thinking that you don't fit in, I encourage seeing that as a positive. Look at that as your unique value proposition and say, what can I do with that? How can I share that uniqueness that I've got?
Daniel:
Have you always found it easy to be comfortable with being different?
Denise:
No, not at all. It's quite funny actually, because people see me now, and see my daughter, for example, who's a highly, highly social human, and make that comment that the apple doesn't fall far from the tree. The truth is, as a teen and as a kid, I was absolutely the one that hid in a corner and didn't want to socialize, because I didn't feel like I fit in. I'd say that it has been a learned art that I've been able to overcome because of the supportive people I've had around me. I do count myself incredibly lucky to have worked with people like Bruce Thompson. There was another manager there, Elizabeth Thomas, who I worked with for most of those 12 years, as well. they were just so good at amplifying people's uniqueness.
Over that decade I really had the opportunity to be supportive of myself and to get that confidence by being pushed into situations, like public speaking, which used to make me feel physically ill before. I did it. Now, I do it without even thinking. I do know that you can learn that, and I do know that you can learn that confidence. I do count myself lucky to have had amazing people around me that have been encouraging. As a mentor, myself, there is a feeling like you are paying it forward. I've had this great experience and had these great people supporting me. Therefore, I feel a bit of a responsibility to make sure that I try and offer that to others, as well.
Finding Your VoiceDaniel:
It makes complete sense, the idea of paying it forward. How do you approach public speaking? There's a lot of people that would really love to have a voice, but they're terrified. They are afraid, maybe they don't know what they like to talk about, maybe they've never tried it before, maybe they're afraid that they're not enough, maybe they're just afraid of being seen. What can we do to get better at this?
Denise:
How did I learn it? Honestly, by having people like Bruce say to me, you're going to go and do this, and I'd go, I'm going to go and do what? Instead of really fighting it, going, okay, well, actually I am going to go and try this. One of the things about public speaking that I would say is it’s like riding a bike. It's frightening when you start doing it, and you feel all wobbly, and you feel like you're going to fall down, and you probably will a couple of times. I think you just have to recognize that's part and parcel with doing that. You'll make mistakes. You'll walk off stage. You'll go, why did I say that in that way? That's just part of it. What I would say though, is that it is like riding a bike in that it does get easier the more you do it. The more you practice, the more you'll read faces in the audience, and see how people respond to the way you talk about things.
It's so valuable to watch how others speak that you enjoy, so spend time on things like Ted Talks, if you like that style. Spend time really looking. Next time you watch someone give a public speech, make a note to yourself of what you liked, and what you also thought they didn't do well. It's just as important to work out what doesn't work, as it is what does work. Remember that public speaking is something humanity has done since we were able to speak. We are a storytelling community.
Instead of thinking of it as I'm getting up on stage with my PowerPoint to deliver a presentation, sit back and actually go, well, actually, what I'm really doing is getting up to tell a story. If you think about most stories, they've got a beginning, a middle, an end, and a moral or something that you are trying to get across as a point within that. If you think about your presentation more like that, you're far more likely to be comfortable and conversational with your audience, and potentially a lot more engaging with what you're doing.
Daniel:
Do you think public speaking has had any major impact on your career?
Denise:
In the type of career that I have, absolutely. I think that these days, particularly working at that international scale, you've got a globe to try and talk with. Because of the technology we've got, so many of the presentations these days are recorded, and so much more accessible to people around the world. For your individual brand, public speaking has a huge impact on people knowing who you are, what you are capable of, and how to connect with you as a person. If you are an expert in a field, I think that public speaking gives people a way to see you as a person. There's an element that that role plays in helping to build trust in leaders that are around the world. People will really look at how someone delivers a presentation, but when you think about having a conversation with someone, how someone speaks to you is a big element in creating trust between you.
What Leadership Looks LikeDaniel:
First of all, let me say, I completely agree with everything you said. It's different from writing a blog. You are on display. You, as a person, are standing in front of other people and saying, here I am, this is what I think, and trying to communicate a message to them. You're on the hook in a completely different way than you are when you write an article, or when you post on social media. It's more intense. I think that because of that, the rewards are much greater. There's a lot of great things that come with public speaking, with having a voice, with showing this leadership. There's also a lot of responsibilities that come with it as well. I have a few ideas about responsibilities that come with leadership, but I'd really like to hear what you think about that.
Denise:
I would agree with that completely. I think that public speaking is about having a voice. Leadership is about having a voice too. There's an awful lot of different styles of leadership. For me, responsibilities-wise, my style is probably more like that of Bruce. I know that I'm only as good a leader as the team that I'm working with, or the group, or the community, or what have you, that I'm trying to speak on behalf of. I know that responsibility-wise, when it comes to geospatial, that I often walk into situations, whether it's the UN, or the World Bank, or World Economic Forum, where my responsibility is to be not just a representative for my organization, but as a representative of our whole geospatial profession.
It is quite a big responsibility for people who have a lot of those leadership roles to realize that you represent more than yourself. It might be just you that's in that room, but often you are representing a much bigger group of people. Whether it's the work that they're doing, whether that's the belief system that they have, or the culture and background that they've got, there's a great level of responsibility for you, as a leader. For me, leaders really have a great responsibility to not just represent that group of people well, but to also look at how they might help them grow. You are in this leadership role of power, which means actually you have more avenues, more opportunities at your fingertips, so to speak, than a lot of people that are working in different roles within the organization you are leading. You have a responsibility to look at that diverse group of people you are representing and say, how do I take this power and opportunity that I've got, and help share that to the broader audience? That's probably going to be good for me as a leader, but it's also probably going to be really good for our organization if my staff, and the staff within our organization, are being the best they possibly can be.
Daniel:
I think you managed to summarize a lot of my thoughts around this amazingly well. Thank you very much for that. It sounds like you've been really fortunate. You've had this amazing leader, this amazing mentor, really early on in your career. For those of us that haven't had that experience, that don't know what good leadership looks like, help us understand if we are stuck in bad leadership. What does bad leadership look like?
Denise:
I think bad leadership is when you've walked out of a meeting with your manager, or boss, or what have you, and what you are doing in your head is feeling like you are not worthy, feeling like you are not able to do your job, that you are always getting things wrong, and feeling unsupported. If you're coming home at the end of the day, and your position or role in your organization isn't bringing you some joy, or you don't feel like you're growing and doing things that you want to be doing, I think you really have to then look at the leaders that you've got that are responsible for you, and evaluate whether they're doing the most they can.
Sometimes I would say that they may not even realize that they're not giving you the support that you need. There is an encouragement I would give to people to say, if you feel like your leaders are not giving you what you need, first try to have a conversation with them. If you can have that confidence to say, I'd really like to try growing in this direction, or I'm really interested in these other opportunities that I've seen in the organization, or I've got a skill area that I don't have, but I'd love the opportunity to try and build that. Is that something the organization can support me in doing? Most good leaders will hear that approach and if they've got it within their power, will try and then help support you to achieve and to do that move. Sometimes it can be as simple as trying to create a good conversation, and sometimes your leaders need a little help to be better than what they are. For me personally, feedback's the only way you can do that and be able to get some sense of where you as a leader need to change.
Daniel:
It sounds like this could be a pretty difficult conversation for both parties involved. You talked about feedback there. How do we give good feedback? Or how do we request feedback in a way that doesn't sound like we're complaining, or that we are entitled to something? Do you have any advice for us there?
Denise:
I think with feedback, first of all is about building a relationship of trust with somebody. You can't just walk up to someone and start critiquing what they've done, if you don't have a prior relationship. Perhaps, that's saying to your manager, or your CEO, or whoever it is, have you got the time, could we have a coffee? Can we have a conversation about something else, so that we can establish some sort of relationship that is going to enable us to have a better conversation? That goes with any relationship, whether that's work or otherwise. No one is going to receive feedback from someone else, if there isn't trust within that relationship that's there. I think that's really important. It can be a bit of a process in developing that relationship, but I would say that's a really good skill to develop in life in general.
Building relationships that are based on trust and authenticity will always create a good space for positive and reinforcing feedback. What I would say about the feedback is that it can't be about you complaining. It's not about saying to somebody, you don't do this for me, and I really don't like the way you do X. There's a lot of psychology all about this, there's a lot of books you can go and read. What is really healthy, and a good way to provide feedback is to talk more about the impact, perhaps, of someone's behavior, or the impact of what's happening in the organization on you. Talk about it more from the perspective of how it impacts you, your work, or in your career progression, etc..
Rather than try to lay blame, look at the behavior and activities of the person and see- how do I phrase this so that it's much more about how I'm feeling, or how what is occurring is impacting me. You'll find that approach will often be a lot better received from people than if you critique their behavior and their actions. In that case, most people get defensive and feel like they're being criticized.
Inclusivity in GeospatialDaniel:
By the time this is published, it'll be pretty close to International Women's Day. I'm curious to hear, do you think that you as a woman have faced any particular issues, when you think about your career? When you think about the career path that you've been on, and these leadership challenges that we're talking about.
Denise:
Yeah, absolutely. Particularly in the early part of my career I would land myself in, say, I could think of a particular conference that I walked into, where I had put on a very nice, very professional, red dress, and walked into a room of 200 males, all in black suits. Then there was me. Blonde hair, a red dress, and heels, feeling like I stood out like a sore thumb. It was a useful thing for me as a female, and served as one of those unique light bulb moments. I think, for many women, they would've walked into that situation and immediately gone, oh my God, I feel out of place. Despite this, I had a great conversation with someone while I was at that conference who went, oh my God, it’s so easy to find you, because you look different from everybody else. It was a helpful thing for me to realize that sometimes being the odd one out actually is a highly valuable thing. You become memorable, you become unique.
One of the things I'd say to women, because in technology I think that's still very common to be one of the only one or two women there, don't try and be one of the guys. I've seen that time and time again, where because it's a male dominated culture, women try to dress more like men, and try to behave more like men that they're dealing with. As a result, they will not call out the culture, behavior, and gender-based jokes. Women, I would say should almost revel in the fact that you are unique in these situations, don't be scared of owning and being a female in those environments. That would be one of the things that I've really noticed over my career.
I've certainly had many of those instances where I've come up against a gentleman who has looked at me as a woman, and as a mother, and gone, essentially, what are you doing here? I don't understand how a woman is in your role with your type of experience. I think one of the things that's shocked me a little bit more was the occasional instance where you end up with other women, particularly women who've risen to high senior leadership roles who are quite competitive with other women, more so than they are with men. At times, that's to the detriment of us achieving better diversity and better equality in what we are doing out there. As a female, I look at other women, I'm like, how do I amplify, elevate, support and encourage? In my head space, I can't even imagine the idea of competing with another woman. I want to bond with them more and say, how do we work together? I think that element of competition from some women I found to be quite a surprising and startling thing in my career. I suspect it's probably built off of the fight that they've had to have to get to the roles that they're in, and the challenge that being a woman can be at times. I think we're getting past that, but it was an interesting thing to look back on my career and realize those moments where that has happened.
Daniel:
This is almost an uncomfortable conversation to have, because as a white male, I feel like when we talk about diversity that I'm constantly part of the problem and never allowed to be part of the solution. Maybe you can give me some advice here. What would you say to someone like me? How can I be part of the solution, when we think about diversity and inclusion in geospatial?
Denise:
I'm so glad you raised that. It's probably one of the things that drives me the most mad these days. I am, without doubt, a feminist. I'm all for equal opportunity. It's that word equal, that we're beginning to sometimes not see. For me, if we've got sections of our populations, and I specifically think that white male part is experiencing this, we've almost tipped the balance too far. If you are feeling like you can't be in the conversation, then we've missed something here, because that's not what this should be about. What we need to do is get to a point where we have a much more inclusive dialogue. We're simply embracing difference, and we see that as a valuable thing to have.
We are embracing the fact that people have different perspectives, and different ways of being, and we're not judging all the time. What I would say to a lot of men is, firstly, I'm so sorry that you feel like that. It's really not something that I would ever want to see happen. What I think is really important for people that feel like that is to look at your interpersonal relationships and how you talk with other people, or what actions you can take within your workplace, look at the simple things. Even if you don't feel like you can have an entry point into the bigger conversation there are lots of small things. Be careful of the language that you use when you're writing a job description and try to put it together thinking about what diverse and inclusive language is.
Familiarize yourself with the tools that are out there these days around diversity, equity, and inclusion, so that you can see those pitfalls that people fall into. Language-wise in particular we've got a whole bunch of cultural references that we use that are still very gender stereotyping. If you can just recognize when you use them, for example we'll see, little girls are described as bossy, whereas the same little boy, exhibiting the same behaviors, would be described as a leader. Being able to recognize when those things occur and when you're using language that's just so culturally complacent, and going, okay, I'm not going to phrase it like that. I'm not going to deal with it like that. That's a really good first step, because everybody should be doing that, not just white men. That's something everybody should be looking at doing and being inclusive.
If you are someone that's in a leadership role, you have that same responsibility. What can I do that's going to help amplify, not just women, but everybody? How do I cultivate diversity in the teams that I'm working with? How do I help create an open and inclusive culture in what we are doing, and help to demonstrate that there's value in doing that? Talking about Bruce Thompson, as a leader, he was a white male, but, my goodness, in the office that we were in, we had women, men, old people, younger people, people from different ethnic backgrounds. Our office was built off of so many different cultures, and he didn't make a point of it, he just lived it as a role model in that diversity and inclusion. I think sometimes it really is challenging to get engaged at that big macro conversation level. There are smaller, more personal things that you can look at that help us move in the right direction.
Preparing for Leadership Daniel:
Thank you very much for that. I really appreciate your insights on that topic. You talked a lot about leaders there. That they have a clear responsibility. I think we all have a clear responsibility to make things better, in general, to look at things that are broken instead of walking past them, to pick them up and try and fix them. I think that's our responsibility. Would you have any specific advice to young professionals, young female professionals, especially, in terms of their career, in terms of they're looking to move into leadership roles? What should they be thinking about today, now?
Denise:
I would say watch the leaders around you, and evaluate in your head, what you like and what you don't like. Watch how leaders behave, watch how they talk, and really sit back and go, how does this make me feel? Does this help me? Am I inspired by what they say? What would I emulate, if I was going to be a leader in this space? That role modeling element is really important, it helps you to pick out bits and behaviors that you do and don't like about being a leader. It will help inform you as to what sort of a leader you want to be. What I would say to young women, or to anybody really in the early stages of your profession is, once you've got a bit of a sense in your head of a leader that you like the look of, go have a chat with them and see if they can be a mentor.
If they're willing to have a coffee with you, ask them how they got to that role. What was it around them that helped build them into the leader that they are. You'll probably find some incredible insights about twists, and turns, and experiences that they've had over the course of time. I know, these days, a lot of leaders, really good leaders, that I work with are often are very open to having that half an hour cup of coffee, or chatting over Zoom. I probably have about two conversations a week with young people from around the world who just ping me on LinkedIn, or are introduced to me by somebody else, and they just want an opportunity to talk to somebody who's been through some of those steps. Don't be afraid, in your head, to critique, and certainly don't hero worship people. Look for their flaws too. We're all human, so no leader is perfect. Whatever leader you will be, will be your style of leadership too. Don't be afraid to just be you, and develop leadership your own way.
Daniel:
Speaking of mentors, or role models, is there anyone out there today where you can point to and say this is leadership? I think they are doing leadership in a fantastic way. This is worth copying.
Denise:
That's a hard one. There's a lot of them doing a whole variety of different things out there. Probably some of the leaders that you don't necessarily see at the top. They're not always CEOs for me. I think that there's a lot of really brilliant leaders that sit within smaller organizations. Going to Women in Geospatial, Julia Wagemann has been an absolutely sensational leader for movement, and really took an opportunity to take action, used her network, used her passion, to really help build an organization that was about helping others. I really hold her up in quite high esteem for her leadership skills.
I think going further, another woman would be Barbara Ryan simply because she's such an authentically passionate person about the use of geospatial, but the use of it for the positive and right reasons in the world, and to help the environment and making the world a better place. She's really a bit of that servant leadership style for me. She's been so much about building community, and never talks about herself as being a senior person. She always talks about herself as being part of a bigger and broader community. Really good leaders don't elevate themselves, or really even talk about themselves as being leaders. They just intrinsically behave like them, if that makes sense?
Daniel:
It does. I really like the example of Julia there with Geospatial Women. I think people that don't wait to be given the opportunity to do leadership, but take the opportunity to be a leader, I think, that's really impressive. That's something we should be celebrating.
Denise:
Yeah, I couldn't agree with you more. I think the reality is, to be a leader is to take risks. There will always be people that don't like your leadership style. That don't share your passion, or your interest areas. You do take a risk when you decide to put yourself out in that way, or take those different roles. I think that it's quite a courageous thing for a lot of people to step up into those spaces. You've got such an amazing opportunity to help change the world by taking leadership in your passion and mission areas,.
Changing Our View of LeadershipDaniel:
Yeah. I think that's a really important thing to think about. It is a risk, because regardless of what you do, it won't be for everybody. There's a really interesting balancing act here, where we need to constantly learn along the way. This didn't work, fix it, make it better, do something different next time. Always remember that this is not for everyone. This is for a very specific group of people. I think it's super important for leaders to be thinking about, well, who am I trying to serve? What are they saying about it? It's okay if there's a person over here who doesn't agree with me, and I'm not trying to serve them. Don't listen to them, listen to these people here, and just be very laser focused on who it is you're trying to help.
Denise:
Yeah, I couldn't agree more. You touched on a really interesting point that has been something that's slowly been emerging in my brain in the last five years or so. Leadership isn't for everybody, and that's perfectly fine. I think we have this strange culture around our career plans, and career progression concepts, that are always pushing this idea that success is reaching the top of an organization. That's something that we really need to challenge as a concept, because I don't think everybody needs to be a leader. You imagine if the whole world was all trying to lead, we'd never get anywhere, it'd be a complete mess.
When we look at our careers, some advice is, don't get caught up in that hype. That the only way you are going to be successful is if, one day, you are the CEO. If you really want to be at that level, and that's something that you know is going to bring you fulfillment, by all means, chase that dream. However, if that idea actually scares the daylights out of you, and what you instead love is to be the really good technical person that's deeply into your particular domain area, creating brilliant maps, and brilliant solutions. If that brings you insights to your passion, and brings you joy, pursue that. See that as your success metric, instead of going after those roles. We put a bit too much stock, at times, in the titles that we associate with the idea of leadership. Sometimes that's to some people's detriment, and I think they end up giving up things that they're really passionate about and, perhaps, really skilled at, because there's this strange expectation we have that success is about being at the top.
Daniel:
I completely agree with that. When I think about leadership, I'm less likely to think about leadership in terms of that traditional understanding in an organization, where someone gives you a title and says, please do this job here. You are now a leader. I think of leadership that happens at every level of the organization. People are trying new things. They're voicing their opinions. They're having ideas. They're seeing opportunities to fix things and make things better. For me, that's all leadership, because it requires that you see something, you have an idea, and you act upon it. That's a style of leadership, and I really want people to understand that is an option. I'm not pushing for leadership as such as the only metric for success, but I'm a strong advocate for people pursuing leadership.
Denise:
Absolutely. I love that description, Daniel. For me, that's also what I see as leadership. I'm not sure that perception of leadership is necessarily shared by a huge number of others. If we can do that, hopefully that will encourage more people to raise their voices, and to offer themselves in leadership, and to take those actions.
Daniel:
Well, we'll just have to continue making podcasts like this, and maybe the message will sink in.
Denise:
I like your thinking.
Daniel:
Denise, thank you very much for being here with me today, I always enjoy talking with you. There will be people listening to this going, well, that woman is incredible. How can I reach out to her? How can I connect with her? If that's the case, and I know it is the case, how can they do that? Where can they go?
Denise:
You'll be able to do a Google search on Denise McKenzie and geospatial. You shouldn't have too much trouble in locating me from that perspective. But always find me @spatialred, all one word on Twitter, so you can find me there, or on LinkedIn, Denise McKenzie is a really easy way to get a hold of me, or in my new role now at PLACE. You can find us on our website of thisisplace.org, and you'll find my staff page there, and the way to get in touch with me from that too, so lots of different avenues.
Daniel:
Once again, Denise, thanks so much, really appreciate your time.
Denise:
Thanks, Daniel. I really appreciate it.
Daniel:
I really hope you enjoyed that episode with Denise McKenzie. As always, there will be links in the show notes to how you can reach out to her, and places where you can connect with her. I really hope you take the time to do that. You've just spent 40 minutes listening to Denise, and I hope we can agree that she's remarkable. She's had a remarkable career, and she is a remarkable person. What I really want you to see is that you can be remarkable too, and I think the only thing that's stopping you from being remarkable is realizing it.
That's it for me. That's it for another episode of this podcast. I really hope you'll take the time to join me again next week. As always, you can reach out to me on social media. You can find me at MapScaping on Twitter. You can check out our website, mapscaping.com, there's contact information there, or connect with me on LinkedIn. Okay, that's it for me. I'll see you again next week.
Daniel:
Hi Michael. Welcome to the podcast. Thank you so much for taking the time to join me today. I really appreciate it. So in just a second, we're going to talk about open-source geospatial journalism, and we'll dive into exactly what that means in just a second for the listeners. Before we get there, can you just take the time to introduce yourself, and perhaps explain how you got involved in geospatial and journalism?
Michael Cruickshank:
I'm Mike Cruickshank, a journalist, originally from Australia, and I actually got into doing geospatial through journalism, which I suppose is a strange entry point to geospatial. This happened because I got involved in this kind of journalism called open-source intelligence. This uses a lot of geospatial data for cross-referencing things by using techniques such as geolocation.
This first brought me in contact with satellite imagery through Google Earth and that sort of thing. I just became more and more interested in what this data is, how can I use it to do more than just have something to look at? What stories are there that kind of lie beneath the pixels, so to speak, of the data and how can I use it in more interesting ways? From there I taught myself how to program, and work more deeply with this data.
I've learned how to use all these different geospatial tools and programs. I'm also doing academic research now, I'm doing a master's thesis where I'm using this kind of data to investigate links between climate change and conflict. I’m also working professionally at a company in Berlin called LiveEO, where we use geospatial data to provide information for industries on things like environmental issues or forestry, as well as monitoring industrial assets. It's quite a broad range of things that I'm involved in these days.
The Link Between Journalism and GISDaniel:
It sounds like you're moving away from journalism and into geospatial. What kind of skills can you already see that are going to be really helpful in that move, that you'll take with you from journalism and over to the geospatial side?
Michael:
I wouldn't so much say I'm moving away from journalism as I am trying to tell a different kind of story to a different kind of person. In the past, my thinking was all about how do I inform people about what is going on? I came to the conclusion that, while I still want to inform as many people as much as I can about what is going on, I also want to be able to reach more of the kind of people who are actually making the decisions in the world, and provide them with information on how things will go into the future.
I'm particularly interested in looking at the effects of climate change and its security implications. I want to be able to provide insights on this prediction, analysis, etc., using geospatial, as well as some of my background in journalism on thinking about the real effects of these things. I want to present this kind of information in a way that's engaging to people so they actually take it seriously, and just reach in a more targeted way the kind of people who can actually make a change in this world.]
Daniel:
You had this great sentence there, you said something like, "I want to tell different stories to a different kind of people." What was it about traditional journalism that you felt like wasn't working, or could be done better?
Michael:
I felt that in many ways, people were flooded by the same story. I was working in conflict journalism so perhaps you could go to a country, you can go to the front line of their wars, you can tell a story of, oh, it's very awful. It definitely is awful what is going on in these places, but the unfortunate matter of fact, is that so many people are telling these kind of stories, people just tend to switch off.
You need to find a different way of reaching them with a different kind of story, one that engages them in a different kind of way, and perhaps tells a bigger picture or a different picture.
Daniel:
I think you're absolutely right with that. I think that's an absolutely brilliant insight. I guess that the danger is when we think about engagement, we sort of fast forward this trend towards the short, incredibly clickable link bait. I really do feel like we are a world trending shallow, as opposed to deep. We see this in all the social media trends, generally. It's quick, short snippets. Do you feel like that's the way to go? If we think about engagement, the numbers speak for themselves. That seems to be what people want, what people engage with. When I think about journalism, I kind of expect something deeper.
Michael:
You're well within your rights to. I certainly think that going after the kind of engagement that you are talking about is a bad thing for publications and for journalists to do. I suppose when I talk about engagement, I don't mean someone just reading something, clicking it, sharing it, liking it, reacting to it. What I mean is someone who's really taking stock of what is written, what is seen, etc., and thinking about it and incorporating that knowledge into their life and their decision-making.
Daniel:
So I have to ask here, we are overwhelmed with media, we are flooded with information and data. How do you think your media, your style, this idea that you are going after here is going to cut through the noise?
Michael:
I'm not going to put myself up on some pedestal and say I could solve this problem on my own. What I think is it forms part of a body of information which is more verifiable, and forms more of a ground truth, or a reality of what has happened or what is going to happen, rather than just 1,000 different conflicting reports from different directions within the political sphere or different interested groups, etc. I think that part of the way that we combat this flood of information is simply providing people with better information, rather than just increasing the amount of it or telling them to consume less. Instead, we should suggest and provide the option for them to just consume better information.
The Tools of the TradeDaniel:
Yeah. I totally agree. I feel like that volume game that you're talking about inevitably leads to a lesser quality of information, or lesser quality of work because it's a volume game, right? It's about getting the amount out there as much as possible.
I know you've come a long way in your journey with geospatial, I know that you program in a couple of different languages now. Could you explain to me what it was like right at the start? Were you using tools like Google Earth just to look at the images and see what was on them and try and geolocate stuff there, or were you using something completely different than that?
Michael:
No, I was using Google Earth or Google Maps, those sorts of platforms, and just purely analyzing things on a visual level. There was no sort of looking at the data behind it, it was just all about identifying things that appeared in pictures, using video locations. I was using platforms like Google Earth, Wikimapia, and a few others to get different kinds of images or different dates of images. Beyond that, I hadn't gone particularly deep into it.
Daniel:
Okay, so that's where you came from. You started off using these very simple visualization tools like Google Maps, I believe you said. Where are you now? What tools are you using and where do you get your data from when you think about doing open-source geospatial journalistic work?
Michael:
These days I'm using a large number of open data sources, but I think the primary and most useful one is Google Earth Engine, which of course is very different from Google Earth itself. Google Earth Engine, as some of the listeners might be aware, is a multi-petabyte catalog of huge amounts of open-source satellite imagery from sources like the European Copernicus programme, Sentinel satellites, NASA's Landsat program, and many others. The beauty of Earth Engine is that it allows people to basically program with the data. You don't have to download these very large raster images and all that to your computer, you can work with it all remotely. The processing is done off your computer. So this is quite powerful because it means that you don't have to mess around with huge amounts of files and huge amounts of processing locally.
Daniel:
So maybe I made a mistake. I've been saying open-source geospatial here, Google Earth Engine is not open-source, even though some of the data might be. Are there any other tools you've been using, or have I made a mistake on my end?
Michael:
By open-source, in this case, I was referring to the data itself. Earth Engine is certainly not open-source, and you can only use it for non-commercial applications, as far as I'm aware at least. In terms of fully open-source programs, I use QGIS as well, so when I'm doing more visual-related output, more than processing, more than programming, more than analytical stuff, when I'm doing visual, I will use QGIS. For the more programming side of things, I'll usually use Google Earth Engine.
Daniel:
So let's stay with Google Earth Engine for a second here. What are you doing? I know you can make these amazing time lapses in Google Earth Engine. What kind of analysis are you doing in there?
Michael:
The kind of analysis that I'm doing, really depends on the use case. Basically, you can do any kind of transformation of the data. If you imagine these satellite images are effectively raster arrays, or just data points, you can do any kind of mathematical operations on the data. You can compare bands or images over time. You can create custom visualizations, and programs that just access this data and then work with it in other ways. You can create front-end and backend applications of this data. Recently, for instance, I wrote a small web program that enabled users to work with Sentinel-1 imagery of some of these Russian military bases where they were building up troops before they invaded Ukraine.
This was taking data from Earth Engine, specifically from the Sentinel-1 satellites, and then creating time lapses out and creating different kinds of visualizations out of this that were presented in such a way that a user could do this without any background knowledge of how to actually interpret the data. Often, especially with synthetic aperture radar (SAR), it's very difficult to get down to what you're actually looking at unless you are experienced with working with that kind of data.
Using Geospatial Intelligence Techniques for JournalismDaniel:
When we say Sentinel-1, are we talking about the SAR band?
Michael:
So Sentinel-1 is actually two satellites. One of them is currently non-operational. I'm not sure whether it's dead or just not working currently. So it's two satellites and they're both synthetic aperture radar, SAR, satellites.
Daniel:
What were you doing with this data? What kind of analysis could you do on SAR data that would give you information about what's happening with the Russian military in this case?
Michael:
The resolution of Sentinel-1 is quite low, it's approximately 10 meters per pixel. Obviously, you can't see something like a tank with this; a tank is smaller than a 10 meter square. If you know the areas where the tanks are being stored, at these bases, you have the polygons of the base. When more vehicles move into the base, the mean reflectance in the synthetic aperture radar increases. You can plot the mean of a time to get an idea of whether the number of vehicles in the base is increasing or decreasing. Moreover, if you have high resolution optical imagery, you can get a baseline of what a certain level of mean reflectance represents in terms of actual real numbers of vehicles.
Daniel:
How do you ground truth, something like that? Or can you use any other data sources to get an idea of perhaps a more precise number of how many tanks, in this case, that there are in an area? Can you incorporate other data sources into this kind of analysis?
Michael:
Absolutely. In my case, I have been using very high-resolution optical data as a ground truth that was taken on the same day. Obviously, this is not always possible because of clouds, especially in the more recent months in winter in Europe, it's very cloudy. You have to kind of wait until you catch a break. This usually can be done with a very high-resolution optical image.
Daniel:
How do you document this, and what are the results of this kind of journalism? Has it been cited anywhere? Are people using it as evidence anywhere? I guess what I'm looking for is- are people referring back to this, almost like a peer review?
Michael:
Looking at these Russian military bases, this was getting a lot of play in the media. I wasn't the only one doing it, so obviously I'm not going to take all of the credit for this. There were a large community of people within the open-source intelligence community who were sharing this kind of analysis. Many of the initial stories about this buildup were coming from within this community, and then were taken up later by the more mainstream media who were then tasking even higher resolution satellite imagery for their stories.
This really did get picked up in the lead up to the invasion. It helped in many ways to lend credence to what countries like the United States were saying when they were saying, "There's definitely going to be an invasion. This is how it's going to happen. They have all these troops here." The advantage of having people doing this kind of open-source intelligence journalism, using things like geospatial data is that we can then have a second data point and say, "Well, they're saying this, but now we can check if there really is data to back this up." In this case, there certainly was, and we've seen what's happened since then.
Using Social Media as a Data SourceDaniel:
Twitter is my social media drug of choice. At the moment, my account is flooded with images and short video clips of tanks and military operations in the Ukraine. At the same time, I'm constantly wondering, is this real, this thing that I'm looking at? Does this make sense? Is it misinformation? Is it disinformation? Can I believe this? Maybe before we dive into this as a topic, perhaps you could explain the difference between misinformation and disinformation.
Michael:
I'll start with misinformation. Misinformation is when someone shares a piece of information that is untrue, however, they don't necessarily know that that information is untrue. More often than not, they will probably believe that it is true. Disinformation is the case when someone is maliciously and with intent sharing something that they know to be untrue. The differentiation lies in whether the person sharing knows whether what they're sharing is true or not.
Daniel:
So this has been a huge problem, obviously, not just in the current situation we are in now with Russia and Ukraine, but also in other political theaters around the world. Is there any opportunity here to sort of prove or disprove some of this kind of information that we are seeing in social media feeds, for example, by doing the kind of journalistic work that you are doing? Or can we use this data together with what you are doing?
Michael:
Absolutely. Using open-source intelligence is one of the best ways that we can actually prove or disprove whether something is indeed misinformation. The primary tool is geolocation, where you can compare elements within, say, a video or an image to satellite imagery to work out whether it really is where it is said to have happened. This often makes it very easy to filter out if something is misinformation or disinformation. You know something is misrepresented, because you can immediately see that, okay, this is indeed where it's said to have happened, which then lends credence to the idea that that is indeed true.
You can take geolocation further and move on to chronolocation, where you start using things like elements of the shadows within the videos and backing this up with geospatial data on the exact position to calculate the exact time that a video or an image was taken. This further leads you down the path of whether this is true, or something which is misrepresented.
Daniel:
I can definitely see the power of geolocation here, but how do you do that? Are you looking for a place name in the video? Are you looking at some sort of metadata in the video, perhaps the geolocation tag, that kind of thing? Or are you doing something completely different to locate that image or that video?
Michael:
If there is a name, place name, shop name, or a business name within the video, this obviously makes it very easy. More often than not, that's not present, or the video has such low quality that it's difficult to actually read what's written. The more common way this is done is by cross-referencing visual elements within the video to visual elements within satellite imagery. This would be things like the architecture of the buildings, the specific layout of streets, of plants, and that kind of thing. (Much like Geoguessr). You then think about what that same scene would look like from above, and then cross-reference that scene with satellite imagery of the location where the video or image is said to have taken place. You then see if you can work out exactly where this happened. More often than not, you can work out not just where, but what angle the camera was pointing in, what time of the day, what time of the year, etc..
Daniel:
This reminds me of trying to orientate myself to a map. Looking at a map, looking for features that I can identify on a map, and then looking up in the real world and saying, okay, where are those features relative to me? Can I use them to orientate myself if I'm out hiking, for example? So this makes sense to me, but we live in a time of deep fakes. You will have seen these videos on the internet before. Is it possible to fake a location, when we think about the kind of information we can glean from social media?
Michael:
It is possible to fake a location, or at least fake a plausible looking location. Geolocation is possible 99% of the time in any video. If you can't find where somewhere is, if you can't get a fix on it, you have a pretty strong indication that this may indeed be a fake location. I haven't specifically heard any reports of videos that have been shared from things like conflict zones where the background has been completely faked using these kind of deep fake techniques, but it's certainly possible.
There's a website called This City Does Not Exist that sort of generates these deep fake landscapes, and there's no reason someone couldn't do something like this. It would take a lot of effort, and looking at some of the misinformation or disinformation that's being spread around, most recently in the conflict between Russia and Ukraine, most of it is pretty low effort. This sort of stuff takes a lot more effort and I have yet to see it being used, especially at scale.
Doing the Analytics and ResearchDaniel:
I'm imagining that the power of this kind of journalism is when you can start joining the dots. You see something happening over here, and you can connect it to an event somewhere else, or you can see that things are moving. How do you keep track of these events? How do you know for example that, oh, okay, I'm looking at tanks here now. I've used my SAR data. I can see that the values are changing over time, so there's change over time. I've confirmed this one location using social media, and georeference some of the videos or images that I've seen there.This is sort of giving me a picture in my mind and understanding of what's happening. Tomorrow, the tanks are gone, things have moved. How do you link that with other events, or how do you find where they've gone?
Michael:
I approach it kind of like a scientific, or a proof statement almost. You're trying to prove that something happened or didn't happen, or trying to work out what the limits of what you can know are. You try to create a chain of evidence going from the open data that you have, and then see how those things link together, how you bring this all together, and how each of these things mutually reinforce, or weaken the argument of the other. In terms of your question you asked, how would you know where the tanks have gone to? Well, maybe you've seen the mean SAR values at this base decrease, then maybe a day or two later, you start seeing videos pop up on TikTok of large columns of military vehicles moving around another town, somewhere else in the same region. So you think, okay, well maybe the vehicles are moving towards that area, but you don't know where they're going.
In my case, recently, I created a map of an entire region, Oblast in Russia, just doing basically change detection in SAR and seeing which areas specifically had had large increases within the last week or two. Obviously you get a lot of false positives; you get things like snow melting, lakes melting, or different landscape changes. Some of these places, however, you might see a very large area of change; a polygon that looks perhaps somewhat artificial. Then you can sort of cross-reference this with other data and think, okay, well maybe there is something there. Then you can download very high-resolution satellite imagery, optical imagery of this area to try to cross reference and determine, is this really something which is interesting or suspicious?
Daniel:
I've got to tell you that this idea of using geospatial in this way for journalistic research is kind of new to me. It makes perfect sense though. It seems like a great tool for this kind of application. Is this different from what you were doing when you were working as a journalist? Is this completely different from the kind of work that journalists would generally be doing?
Michael:
I think in some ways, it is very different from traditional journalism. Traditional journalism is all about building human sources, and your strength of your argument, or of the story that you're writing. It is almost based on your level of access that you have to reputable or trusted or well-positioned human sources. That is the skill; it's a networking skill in many ways, it's being in the right place at the right time, etc..This kind of journalism is about pulling together a whole lot of things that everyone knows, or everyone could know, and then seeing how those things all work together to prove something and drawing connections that other people aren't.
This is unlike traditional journalism, where it's all based on trusting that this is a reputable publication that wouldn't lie to you. You've got to trust that this anonymous source is really a real person, etc.. The opposite is true with open-source intelligence. Everything is effectively a proof statement. You don't need to trust the journalist. You can read through the way they've brought together their argument and their evidence and see if you agree with it. There's nothing that's hidden. Everything is in the open.
Daniel:
For me, this begs the question, why aren't big publishing, media, journalistic companies doing this? Or are they doing this? Do they have their open-source and intelligence center within their magazine or their media company?
Michael:
Over the last few years, there have been more and more mainstream media outlets that have been starting to use these techniques. It is growing in popularity, but it's also a relatively new thing. This really only started probably in 2012, 2013. That was when the ball started rolling. It has just grown more in importance over time, and in popularity and sort of the credibility that it has.
Taking the Story to the PeopleDaniel:
I'm pleased you brought up credibility. What do people say when they push back on this, when they don't believe it's a great idea? What kinds of arguments do they come with?
Michael:
It depends. A lot of the time they simply just don't really understand what you are doing. They'll just say, "Well, this isn't forensic enough, this isn't scientific. You're letting your own biases get in the way. Your analysis is wrong." They don't really understand how these things prove each other and so they just say, "Well, you're just hand waving things around and saying that this equals this," and they don't really understand why this equals this, so they dismiss it. To be fair, most of the criticism I've come across is more or less just bad faith criticism in the sense that these are people who have motivated reasoning and start with a position of "You're definitely wrong, because you're saying something I don't like." I find that there's little good faith criticism, and it's mostly just bad faith, politically-motivated criticism.
Daniel:
You brought up a great point there, that people didn't understand the analysis you did; perhaps they didn't understand the techniques, they didn't understand the data that was being used. You're telling people, "Yeah, I looked down from space and watched the change over time." I mean, it sounds pretty fantastic. How do you explain that to people?
Michael:
This is sometimes the harder part, because you have to explain a bunch of things that aren't always intuitive. I suppose for people who work in areas like geospatial, thinking in a spatial manner is very natural to us. For other people, sometimes it's not. Imagining what a scene might look like from above, imagining how things could look over time, etc., or how things can be abstracted in a 2D or 3D format, can be difficult for some people. You need to make this easy, and so having really good visualizations is incredibly important for this. I've seen some groups that make really nice videos that are sort of linking things together with nice effects to sort of show how things transition over time. It's all in the kind of visual communication you use.
Daniel:
Oftentimes, I think that the people that work in geospatial, we get stuck in only doing analysis for big companies or municipalities, organizing and maintaining the data. When I meet people like you that are doing work like this, I think, wow, here is another opportunity, here is something that people should know about, because it makes so much sense. We can use these tools that we know about, that we're comfortable with, and we can use them to tell a story. For me, the skill sounds like investigative journalism that tells a story, and backing it up with evidence. Could you imagine a time where new journalists need to learn these skills just in the same way they need to learn interviewing skills or other media skills, making that data analysis a part of their education?
Michael:
Absolutely. It will be critical going into the future. It's the only way that journalism can push back against the kind of flood of misinformation and disinformation that we're currently facing. This is a perfect way to combat it. Rather than just retreating back to ideas of, well, I'm a reputable outlet, or, you have to trust me at my word, instead, you're saying, no, don't trust me. You're right to not trust me, but here, I can prove it. I think this will hold much more weight than simply asking people to take you at your word. If more and more stories are presented in this way, then it won't be so much that people trust the media more, but the correct information will get out more. In the end, that’s what's important.
Daniel:
If you do a great job of documenting this kind of work, do you think this will help people think critically about how people draw conclusions when they see other types of journalistic work?
Michael:
Absolutely. I think this idea of thinking very forensically, in terms of how you prove something, and what proof really means, and where the edges of what can be proven and can't be proven lie really helps you narrow down where the gaps in our knowledge are, and where the area for debate lies. That way we don't spend so much time talking about things that are kind of extraneous, or are red herrings within the conversation.
Daniel:
Michael, I think we could probably round things off here. I'm curious if you have any recommendations or references. If I want to learn more about this open-source intelligence, if I want to be a part of this, is there a community I can join? Is there a newsletter I can follow? Is there somewhere I can go?
Michael:
In terms of communities, there's a great Discord community called Project Owl, which I think currently has about 25,000 users. It's growing very rapidly with hundreds of people working on all sorts of different projects all around the world, all brought together by this kind of usage of collecting information from open-sources, especially about conflict zones, but also about many other different areas- bringing this together and synthesizing this information and seeing what can we prove? What can't we prove? etc..
In terms of other sources online, I think the best group in the world who's doing this is Bellingcat. They're a UK-based publication started by Eliot Higgins, and they've done some really great investigative work. I believe they've also won a Pulitzer Prize, if my memory serves me correctly. They were looking at MH-17, some of the Novichok poisoning attacks in the UK, they've done all sorts of interesting looks at different conflict zones around the world; in Syria, in Yemen. I've also written a few articles for them myself, looking specifically at Yemen and Ukraine. There are lots of great people associated with this and they're doing really good work all the time.
Daniel:
So I really hope you enjoyed that episode with Michael Cruickshank. I found a few other articles that I want to link to and share with you. And one of them is called The Growing Problem with Deep Fake Geography: How AI Falsifies Satellite Pictures, or Satellite Images. There's another one here along the same lines, When is Satellite Imagery Fake? And the third one, Why Newsroom People Need Expertise in Remote Sensing. I'm also going to include a link to a newsletter on Substack that I found that I think you might find interesting. It's called actualcontrol.substack.com. If I read the headline of this blog newsletter, it says A Blog About Satellite Imagery, Social Media, and Other open-source Information from All Corners of the Internet. There will also be a link to the publishing house that Michael mentioned, called Bellingcat.
Before I let you go, I just want to highlight one of Michael's insights, and that was at some stage during the start of the conversation he said something like, "I guess I wanted to tell a different story to a different kind of person." He used this idea of telling better stories. I think this is really important: telling better stories. We talk about data stories and we talk about customer journey stories, but we never talk about telling better stories. Michael wanted to tell better stories. He wanted to do that because he discovered that the old stories weren't working anymore, they were being drowned out. They were too similar. Everyone was telling the same story so it wasn't sinking in anymore. It was blending in, it was becoming part of the background noise. That sounds really simple, right? Tell a better story. What does a better story look like? I'm not completely convinced you need to know what a better story looks like, or what the right story looks like. I'm more convinced that you need to try a new story. If you try enough new stories, you'll find the right story.
Okay. That's it for me. That's it for another episode of the Mapscaping podcast. I'll be back again next week with a new story. I hope that you'll join me then.
Daniel:
Hi, Jeff. Welcome to the podcast. So you are the director of user experience at a company called Element 84, and you've got this amazingly rich background in design, and now you're working in geospatial. For the sake of context, could you just take a couple of minutes to introduce yourself to the audience and perhaps explain to us how you got involved with design, your background in it, and how that led you to work with a geospatial company?
Jeff Siarto:
Absolutely. I'm Jeff Siarto. I'm the director of user experience (UX) at Element 84. E84 is a small software engineering company that specializes in geospatial and science systems. I got here through a sort of roundabout way. My educational background is not in geospatial, or science. I studied general design and user experience in college. After that, I did freelance web design for a while. During that time, I wrote a couple of books for O'Reilly and their Head First series. In that time, I met Dan and Tracey Pilone who run Element 84. They were co-authors in that same series so I was able to get to know them early on in my career.
Before I came to E84, I started a small social media analytics company, which I ran for about five or six years. We did early social media analytics for consumer product brands. I worked with companies like Wrigley, and New Balance, and Radio Flyer, and we essentially monitored their social media traffic, and then helped their marketing teams kind of figure out messaging and how to engage on early social media channels. That was sort of my initial background. I worked on the software there, and I worked on the design of those projects. After I was done with that, Element 84 reached out to me and asked if I'd be interested in working on this NASA project. I said, "I don't actually have anything going on right now. That sounds really interesting." That's sort of how I got here. I came onto that project knowing very little about remote sensing and earth observation, but bringing my design background to bear on their problems. They had never actually worked with any designers before, so I was kind of new to them as well. I sort of just learned as I went through some of their early projects.
Design and Earth ObservationD:
So we're going to talk a lot more about design and earth observation in just a second here. I'm curious, when you think back to when you started at Element 84, you said that they'd never worked with a designer before. What was it like for you coming in? Were they doing the right things? I guess a lot of the semantics around earth observation was new to you. When you looked at the products that they were making, the things that they were doing, was it all brand new, or could you see a lot of the design elements that you were used to working with, or was it something totally different?
J:
They were getting there. I was actually the first designer at Element 84, and then I was also the first designer to come onto this NASA program, so they had some foundational pieces there. They had great backend systems. They were working in the right direction, and I was able to come in and sort of help them improve the interface, or get their interfaces to the point where they were in the same league as some of the backend systems that they had. They were kind of getting to that point where they had solved a lot of the early problems, but now more of those user experience problems were starting to surface. That has ended up being a trend throughout my career here is once those early problems get solved, those user experience problems sort of creep up and become the major players.
D:
Now that you've been working in the industry for a while, when you think about earth observation and design, do you think a lot of the companies that are doing similar work to what you are doing are facing the same problems? I mean, does it surprise you in any way that we weren’t thinking about design earlier?
J:
I do think that more companies are starting to look at that. As we start to solve those kind of middleware early problems, the piping, and the user interface problems start to creep up. I don't necessarily think that it's surprising. If you step back and look at the timeline of the early internet and where all of that went, design became more important as the web matured. A lot of people sort of moved from that graphic design discipline onto the web. They brought those skill sets to bear in an area that was highly technical before it was a focused design medium.
I think we're seeing that in geospatial, particularly as the tooling gets more web-based and cloud-based, or you're interacting with these applications and APIs through the internet, you're going to encounter more user interfaces. I think the importance of those user interfaces is in making the job of whoever that end-user is easier.
Reducing the Time to ScienceD:
In one of your tweets, I saw this great catchphrase, and it was reducing the time to science. My first question is, how is design going to reduce the time to science? I think design could mean a lot of different things for a lot of different people, depending on what it is that we are working on. If we think about the geospatial stack, we could divide it up broadly into the backend and the frontend. I'd like to start with the backend. You've been talking about it as pipes and infrastructure up until this point. How do we bring design to bear on that? What bits can you come in and design to make it better?
J:
Time to science is super important. To define it a little bit, it is the amount of time that a particular scientist or researcher will spend getting to the answer of the question they initially posed. In the past, scientists have spent 50, 60% of their time doing data wrangling, or data processing- getting all of the bits together so that they can answer the question.
The first part of there in reducing time to science, is sort of smoothing out that backend process. Maybe that's going from having to deal with level zero data, or level one data, up to a more refined data product that is properly subsetted or properly re-projected, and taking care of that for the scientists. Now they don't have to do all of that specialty work on the data, and that gets them one step closer. If you sort of continue on with that, particularly as we move into a cloud data paradigm, maybe we go even further. Maybe we take it and we put these data products into a format that can be easily visualized, or easily manipulated in a way that does require programming. So we've sort of eliminated some of the lower level tasks from the workflow and we sort of shorten that time to science. As a result, now maybe they only have to spend 30% of their time getting the data in the right place.
If you go back even further, some 60% of data wrangling time was just literally waiting for data to download. I would hear stories when I first started where we would go out and talk to scientists, and we'd watch them work on their projects. We'd look at their workflow and see how they interact with the data. Oftentimes, they'd just have to set up a download, and just let it go, and come back a couple days later when all their files were downloaded. Some of that time to science was literally just kind of sitting around and waiting.
Even just improving how you interact with the data, and just generally improving the speed of the internet that people have access to is user experience as well. The speed of the internet access, the way in which they have to get the data onto their machine, and what formats that data is in. They all are related to the overall user experience.
Data Science Infrastructure as a CommodityD:
When I think about the backend, apart from the data manipulation to get it to that analysis ready-state, I think about infrastructure like blob storage and the cloud. I think about infrastructure like cloud optimized GeoTIFFs and other cloud optimized data formats so that we can stream data directly to a client without having to run a database somewhere in the background. I think about things like the stack interface. It seems to me that these pieces are sort of slowly coming together, and that at some stage, they're going to be a commodity. It's going to be “of course you're running with these types of standard components”. I'm wondering how you see that and think about that in terms of being a commodity. Do you think it is at the moment? Are we there yet? Of course you're using these things. What else would you be using?
J:
Yeah, I think we're moving in that direction. I would not say we're quite at a commodity yet, but I think we're getting there. I think you can tell that we're getting there in a couple of different ways. You're starting to see the major cloud players building these systems up. I'm speaking specifically of the AWS public data sets and the Microsoft Planetary Computer. You have these big cloud players that are kind of investing in these open geospatial systems. They are building this backend, this foundational layer of data products and APIs that then can be built on top of in a standardized way.
I think we are in the building stage for that layer. I would say I don't really know where we are in that building stage, maybe halfway through. I think in the decade-ish that I've been working in this, over the last three to four years, you've really seen an uptick in cloud data adoption, and seeing investment in those platforms that are going to maybe become a commodity in the short term. I don't think we're quite there yet, we're still in the building phase.
D:
It sounds like you are thinking in perhaps the same sort of lines as I am- that this will be a commodity at some stage. When I think about earth observation, and I think about software companies that are looking to build a business around it, I think it's going to be pretty hard for them to create the data themselves. There will be satellite platforms which are going to create the data. It's going to come down to earth. It's going to go through a commoditized system, and it's going to end up in the frontend. I think the place where a lot of value would be created, would be in the frontend. Do you think that I'm on the right track there?
J:
Yeah, absolutely. Part of the issue that we're facing right now is that Earth observation and geospatial data is fairly complex to use. We're taking a step in the right direction by moving that to the cloud and creating standardized systems on top of that, and now we've sort of eliminated that initial barrier. Once we have that, now the companies, and the startups, and the government agencies that want to start spending more time answering specific questions can do that. If we've got a commercial interest or issue that we want to answer a question about, now, you have the foundational pieces in line to be able to answer those questions, and focus on taking the data that is in the standardized format and manipulating it in a way to answer a specific question for a specific customer.
All you have to do is sort of focus on what that question is and how to build or manipulate the system to answer it. But you no longer have to worry about “Okay, where am I going to get this data? What data set do I need to use?”. You won't even really need to worry about what the sensor is or what the platform is. You'll have the normalized data ready to go. So I feel like that allows you to really focus on the customer problem.
I think a good example of this is a company like Twilio, which is sort of like a communication and SMS/voice API over a bunch of lower level technologies. If I want to then build a product that utilizes some sort of SMS messaging, or some sort of voice system, I don't have to then go interact with the SMS layer, or figure out how I'm going to do voice over the internet. I can just build on top of Twilio's system. They've sort of standardized out that lower level tech for me. I can focus on my AI chat bot, or my SMS messaging tool or something like that, without having to worry about that slightly lower level. It frees you to think about the higher level problem, as opposed to where you're going to acquire the data, and how you're going to get it into the right format.
The Future of Frontend Design in Geospatial Data ScienceD:
When I think about the frontend opportunity, we have these great platforms we can build on. I think about the brand-ability of a frontend. The way you can dress something up, you can change the look and feel of it. You can add an identity to it, and that can become a selling point in itself. It doesn't feel like those in the geospatial industry are that interested in changing the way we do frontends, and designs. It seems to me that we are pretty committed to using the tools we've always used. Firstly, I'd like to know what you think about that statement. And then secondly, I'd like to say do you see any opportunities to change things here? Any things that could be done differently?
J:
Yeah, I do agree with that. There's a pretty consistent set of tooling that's been used forever. I think there's a lot of standardized patterns that we've seen used quite frequently. I'm kind of talking about a map interface where you can drag around a map, you can create a bounding box, and you can sort of interact with a Google Maps style interface, which has become fairly ubiquitous. Ubiquitous both on the sort of geospatial earth observation side, but then also the consumer side as well. From a geospatial perspective, when you're looking at a map with a bounding box and a sidebar of layers, you're still thinking- “I need to pull data from around this area. Give me all the stuff inside of my bounding box.” I think there are parts of that that are going to carry over. I do think that we're going to need to move into a direction where we are building interfaces that are answering specific questions as opposed to just building interfaces that are allowing me to grab a bunch of stuff inside of a bounding box.
There's always going to be that. In geospatial and earth observation, you're always going to have to tell the system where on earth you want to get data from. I'm not necessarily convinced that it always has to be on a map. I'm also not convinced, after spending many, many years watching folks interact with map products, particularly in the earth observation and earth science world, that that's really the best thing. I think people still struggle with it. I think they're still a little inconsistent, and I think we could do better.
Yes, these tools are ubiquitous. But we still have not ironed out all the UX issues of the map-based interface, let alone moving into a new paradigm. I'm not convinced that it's the best interface once we get into that higher level “what question do you want answered?” phase.
D:
Every time I see a new data portal, it feels like it's a total one-size-fits-all without too much thought that's gone into it. It's a map. It's the panel on the left hand side of the screen. It's all the usual things. Part of me thinks seeing standard products, like standard design, things that I know and have used before makes it easy for me. I don't have to learn something new, so I get that side of it. It just feels like this one-size-fits-all product where no one has really come along and thought- Can we do it differently? Is there something else? We even see Google Maps doing the same thing. It feels like maybe it's not a technical problem. Maybe it's more of a cultural problem.
J:
Yeah, absolutely. I don't think it's a technical problem. There have been great strides on the technical side of internet map making. There are all sorts of great tools to make creating layers and getting the map in your browser easier. I certainly would rather be building map interfaces now than map interfaces a decade ago. We've come very, very far in the developer experience with how easy it is for a software engineer to put a map interface together, but I think that's where we've been iterating. We've been iterating on- How can we make this map interface a little bit easier to use? or a little bit easier to develop? or easier to serve information on top of? But I don't think we've ever really stopped and thought, is a map really the right interface for this particular application?
I think that's sort of where I see us going- Maybe a map's not the best interface for this particular piece of software. And if not a map, then what?
D:
I've often heard people get really excited about having an output they can put directly into a spreadsheet, because that's what they are used to working with. Something that’s going to go directly into a database, or talk nicely with other spreadsheets, and they didn't actually want a visual output at all.
J:
I think that's true. You're kind of always thinking look, I can give you this picture of the land area. We always want to put things into a visualization. Doing visualizations is an important part of this., but not all science users want their output in that format. Sometimes they want it output in code. Sometimes they want it output in a spreadsheet. The flexibility in that output is important. Not assuming that your user wants it in any particular format, but making things sort of extensible in a way that they can pull it into whatever format they're most comfortable with is the way to go. Whether that's a statistical programming language or, like you said, just a standard spreadsheet.
Going back to what we were talking about earlier, with having that middle piece built out and matured, that lets us think about those interface problems. We can spend more mental energy and man hours on those types of problems, because we're not having to sort through the lower level building blocks.
UI/UX Standardization in Geospatial D:
MapScaping used to be an eCommerce website. We ran on a platform called Shopify. Are you familiar with Shopify?
J:
Yes, absolutely.
D:
So Shopify showed up and made it incredibly easy for people like me to start an eCommerce website. There is an app environment, so you can just add these plugins, these apps to your Shopify store from a standardized box. You can quickly make a custom thing, and it's absolutely amazing. It's interesting with eCommerce. Their whole thing is to get someone into the shop and out of the shop as quickly as possible. If the checkout takes more than five seconds, it's too long. There's this real focus on “get the customer what they want as quickly as they can”. I don't feel like we've got there yet with geospatial. I'm wondering if there's any lessons that you can see in eCommerce, perhaps from Shopify that we could take over to the products that we build in the geospatial world?
J:
Shopify is a great example, because they've done a handful of things really well. One of the things that I love about Shopify is that they've sort of standardized the user experience patterns around the checkout experience. When you're checking out on a Shopify site, you know that it's a Shopify site. A lot of people probably don't know it's a Shopify site, but they know that they've seen this pattern before. They know they've seen this particular way the credit card has to be entered into the form. They expect when they start to type their address, that a dropdown is going to appear, and they're going to be able to select their address, and it's going to autofill. They don't have to do all that. That experience is seamless and it's sort of expected. The end user knows what they're going to get. I think one of the problems I see with geospatial interfaces now is that you're not always sure what you're going to get.
My one example of this is the way that an area of interest selector, or a bounding box selector on a map interface behaves. Sometimes, you have to click and drag it. Sometimes, you have to click and add points like a polygon. Once you have the area selected, the way in which you perform actions on that selection are also different. I think there are ways that we can standardize that. I think the group, or company, or people that figure out the best user experience will have standardized it in a way that makes it kind of crazy to not use that. That's one thing that Shopify's done really well is you really have to convince someone to not use the Shopify platform for their store. The people that are using Shopify, they're using it because they don't want to deal with the eCommerce aspect. They want to sell their T-shirts. They want to sell their eBooks. They want to sell their information products. They don't want to have to learn a new system to do that, or put anything out there that is going to make it difficult for their customers to buy their product.
I think with geospatial interfaces we need to move to that point. We need to make it so that there's an obvious choice for some subset of these user actions, but not everything. There are certainly people who choose to roll their own eCommerce site for various reasons, and that's always going to exist. I would love to see some of these patterns mature in geospatial user interfaces so that our science users and our municipal users who are trying to get questions answered for their county or state don't have to wonder what they're going to get when they try to pull data to answer a question. They know there's going to be some standard level of interaction there.
There are two sides here. It's great for the developers because they don't have to reinvent that wheel. They don't have to come up with a new way to do a bounding box, or a new way to do a map interface. On the customer side, they don't have to wonder, "Oh my gosh, this is a new geospatial system. What am I going to get here? What is this going to be like?". I think right now, there's still some of that.
A lot of the science users that I talk to, they're certainly apprehensive about new systems, because oftentimes it takes them a long time to become proficient in the one that they're already using. They do not like to have a new system designed by these new designers that just came in. I'm totally putting myself into those shoes. At one point, I was the new designer that came in and I was like, "You guys are doing all this wrong. You need to do this, and this, and this." That kind of freaks people out, especially people whose entire career is based around doing this type of science work. That specific data is very important to them, both to their career and whatever agency they're working for. There's a lot of uncomfortableness with that change. I think some of that is growing pains, and we've got to get through that.
D:
So we talked about Shopify, and one of the beautiful things about it is that establishment of a standard set of patterns. We know what a checkout looks like. We understand this. Three steps, there's autofill forms, there's this, there's that. Then lastly, we push our credit card numbers in. We get that. Do you think the lack of standard patterns in geospatial speaks to how niche it still is?
J:
There's not as many designers working on specific geospatial problems. I think that's certainly changing. I think a lot of the new interfaces that we see coming out from a lot of the new commercial companies that are appearing, they're taking design a lot more seriously. There just hasn't been a lot of people working on these problems.
There obviously have been great designers working on mapping problems for quite a while. Look at Google Maps, or Apple Maps, or Mapbox. I think on the earth science side specifically it gets pretty niche, especially because of the complex science that's going on there. I think the consumer side is a little more mature, but I'm not entirely convinced that the patterns developed on the consumer side necessarily fit the scientific model or even the municipal GIS worker model. Those are different people that need to navigate in a car or look at an aerial imagery of their house versus a user who's trying to work out land use patterns, or monitor sea surface temperature, or things like that. There are different requirements.
Breaking the Status Quo with New UI/UX Functionality in Geospatial D:
I think this gets back to the idea that one size doesn't fit everyone. We need specialized maps or specialized interfaces, depending on what problems we're trying to solve. I want to stay with the Shopify example just for a second here. In our store, we had this option to install a plugin. That plugin was going to provide a little mini recommendation engine- People that like this also like that. I think in the past, a big argument for having the map was, "We can see the data to decide if that's the data we want." It's an incredibly quick way of filtering a massive amount of data visually. I would love to see some sort of recommendation engines come into these portals to help us filter through it. I would like to see some voice interfaces built into these things. From a design perspective, can you see anything that's stopping us from integrating some of these ideas?
J:
I don't think there's anything specifically stopping us from integrating these ideas. I think we are moving towards that in some areas. We have this notion on the earth science side of “usage based discovery”. This is finding data sets based on what you're trying to do. What is the end goal of the data set, and categorizing things in that way.
I think as the metadata systems, that middle piece that we're in the building phase of now, I think as that gets improved we can expand the metadata available for these data products. Then those alternate discovery mechanisms become easier to build.
Part of the problem is that, in order to build those higher level discovery systems, you still have to have the information about the data. You still have to have the metadata about the products. As we're building this middle piece out, those metadata are becoming more refined, more detailed. Again, because we're not dealing with the lower level problems, people have more time to refine and incorporate more specific metadata into these programs. We can start to build smarter systems that sit on top of that data and allow us to have some sort of AI, or machine learning algorithm make recommendations for data that we may not have thought about. Maybe we have an interface that isn't a map, and you're just sort of asking the system a question based on a region of the globe. Then it can suggest other questions related to that. Or it can say, "Hey, these other researchers have posed a similar question. Here are their results." I think there's a ton of space for those types of interfaces once we have the sort of data infrastructure there, but we aren’t quite there yet.
D:
Is there anybody out there at the moment that's doing inspirational work in this space that you can point to and say, "Well, if you want some examples of great design, great interfaces, look over there"?
J:
Yeah. So a couple examples off the top of my head. I think the folks at UP42 are doing some really great design and user experience work, from a product perspective as well. They're sort of an example of the higher level system that's sort of interfacing with a bunch of these other data providers and then providing an interface or a system on top of that to streamline a workflow to get a faster answer to a question. I really like what they're doing.
I also love this little project called Placemark by Tom MacWright. He's doing some really interesting user interface work in a niche space. Unfolded.ai is another really great one. Their mapping and general design interfaces are really, really good. They're built up on top of Uber's H3 geospatial library, which is a hexagon-based geospatial library. This is being pulled out of the consumer, commercial side, and adopted now into more of the data analytics and geospatial side of the house.
Obviously Mapbox, I'm sure everybody knows about Mapbox. They're doing really, really wonderful stuff. They have fantastic designers. Their 3D rendering and map rendering is really amazing. They are importing consumer and commercial tech into the data analytics and geospatial side. Those are all great examples of fantastic design and user experience.
I'm excited because all of these new companies and projects that are cropping up look great. They're all seeing design as a first class citizen. I think we can just continue to build on that momentum and raise that bar a little bit. Then those expectations for where design and user experience need to be for these systems are just going to keep getting better. We're going to see the same trajectory that we've seen on the consumer and commercial internet side. Pick your industry. All of those systems have been improved with designers having a seat at the table.
Design and/or User Experience?D:
You mentioned design and user experience a few different times there. Do you see them as being two different things?
J:
Yeah. This is like an endless debate. I feel like I was just having a conversation on Twitter with somebody about the difference between design, user experience, and user interfaces. If you're a user experience designer, does that mean you also design user interfaces? And if you're only a user interface designer, are you also a user experience designer? I'm not sure if that's exactly the right argument.
I think I see design as the whole of everything. I consider myself a designer. There are a multitude of practices under ‘designer’. I think a designer could be a pure art illustrator. I would call architects designers. I think industrial designers sort of run the gamut.
User experience for me is the entire end-to-end workflow of engaging with a product. Oftentimes that is not necessarily the way the buttons look, or the way the map looks, or the way the interface looks. It's the process by which a user gets through the system. What are the steps that they have to take to get there? What is the language that you're using in the system? What words did you pick to describe different things? What icons did you choose to represent particular things within the interface? It even goes even further than that. What is it like to engage with your company? So if you're selling geospatial data and you have to send an email to a salesman and then jump through a bunch of hoops to get any information on a data product, well that's part of the user experience too. That's the early, early interactions.
Oftentimes at Element 84, when I'm looking at the user experience, I'm not only focused on the UX of the products that we're developing, or the tools that we're building for customers. I'm looking at the user experience of the company itself. What is it like to interact with Element 84 on social media? What is it like to interact with Element 84 when you're engaging as a new customer? Is the contracting goofy, or is it really hard to get a response from someone? Does the person you had to deal with from a project management standpoint, are they treating you well?
Specifically for geospatial and earth observation stuff, the user experience sort of starts at the engagement with the organization that is brokering the data. What's the UX of going to NASA for earth observation data? It starts pretty early on in the cycle, and our user interfaces are only one tiny part of that overall experience.
D:
So it sounds like what you're talking about is the promise. What is the promise here? When I interact with a company, when I interact with a piece of software, what's the promise that is made? And is the promise kept throughout the interaction?
J:
Part of that promise is that you're setting some level of expectation. You need to be careful. It's easy to talk about a particular feature, process, whatever, and set that high expectation. Then you've got to deliver on that throughout the entire user experience of that individual interacting with your system. That system could just be a little piece of software, or a website, or an entire company. I think keeping that promise is important. When you get into complex systems, particularly in earth science, it is very hard to keep that promise all the way through. There are so many places in that process where you can sort of lose your promise. It's really important to be vigilant about that. It’s the hardest part of the job. Making the interface look good and polished honestly is the easy part. The really hard problem that we have to tackle, is really how do we keep that promise from the initial interaction all the way through to I've got the answer to the question I had.
The Future of UX/UI Design in GISD:
I feel like we've covered a lot of this already during the conversation. When we look out into the future, if you had to sum up things into some bullet points, what do you think we can expect from user interfaces when we think about geospatial?
J:
First and foremost, there's going to be kind of a layering over geospatial. I think we're going to worry less and less about the sensor technology, the resolution, all the tech specs of the geospatial data itself. I think we're going to get to a point where we're not going to have to worry about the low level earth observation data. So there's going to be a pretty good obfuscation of that. I think you're going to get to a point where you're going to get transparent sub-setting and re-projection. It's just going to be like having to understand the intricacies of SMS messaging or any of the HTTP layers on the web.
I really think we're going to be done with downloading data. I think the future data sets are going to be enormous, like terabytes, petabytes. Huge, huge data sets. It's not going to make sense to pull that down over even your best gigabit ethernet pipe. Everything is going to be an API layer. Everything is likely going to be sitting with major cloud providers. You could have a whole other discussion on how you feel about centralization of the cloud providers. That's beyond the topic of this conversation, but I think that's where we're headed, at least from what I can see.
I also think particularly on the earth science side, you're going to get into these low code or no code science tools. There are a lot of really great projects going on right now. Pangeo is one of them where they're building a lot of standardized Python tool sets and libraries to sort of deal with earth observation and geospatial data at a code level. Now we don't have to write this low level code anymore. We can write this higher level elegant Python to do some of the data processing. As we move into the future, I think we're even going to get a step above that. We're going to get to the point where I could be a data scientist or an earth scientist and not have to deal with spinning up AWS instances, or making sure I have a particular Python library. Looking 10, 20 years into the future, I think the low code, no code science tools are going to be the big step there.
D:
Do you think the community is going to push for this? Do you think the pull is going to come from outside of the geospatial community? Do you think scientists and geospatial users are wanting this today, is this going to make their lives easier? Or do you think that it's going to be people from outside the community saying, "We would love to use that, but you're going to have to do these five things first"?
J:
I don't think we're quite there yet. I think the pull right now is “Give me better code tools. Give me better Python libraries. Make this whole spinning up cloud instances go away. Get me out of that.” I think we're moving in the right direction with things like Jupyter Notebooks and JupyterHub standardizing and making it much easier to do data processing at scale in the cloud. So I think we're getting there, and I think we are answering the call to improve developer tools, but we aren’t there yet.
I think that the push is going to come from the other end though. A good example of this, the future scientists, think people that are maybe in high school or middle school now that in 15 years, they're going to be taking over desks at NASA, and NOAA, and USGS. They're going to have grown up on low code and no code tools. They're going to have expectations that the science tools are as easy to use or of a similar capacity as what they've seen in other parts of their digital life. I think the push is going to come from that direction.
In order to get the low code or no code science tools, first you've got to have really great coding tools. The developer experience has to be really, really good, then the developers can stop worrying about the lower level code. Then they can start thinking about, "How can I write this piece of software where my scientist doesn't really have to do any coding at all, and they can just be an expert at land use, or they can be an expert oceanographer without also having to be a software engineer Python expert?"
You could think of other commercial interests full of people that don't have any desire to do code. They're going to be pushing for this too. You see this in commercial web products, like website builders, and even low code or no code tools for building mobile apps and all sorts of complex web interfaces. We're getting to the point now where the infrastructure for writing code on the web has gotten so good, that we are thinking one level higher now. We're able to think about how we can build tools that'll grant that same power to an end user that the developer has without them having to be a software developer.
Daniel:
I think this is a really, really great place to round off the conversation. I want to say thank you very much for walking us through this. I've never talked to a designer before on the podcast. I've really enjoyed the discussion. It's opened my eyes to a lot of things.
Before I let you go, if people want to reach out to you, if they want to ask more questions about any of the things we've talked about today, where can they go to do that?
Jeff Siarto:
The best place to get in touch with me is probably Twitter. They can get me @jsiarto. So first initial, last name on Twitter. Or you can also find me @Element84, which is our company's handle. Both of those places are great. DMs are open as they say.
You don't have to be a big organization to create a business or side hustle around your geospatial skills. If you are looking for examples and ideas ... this episode is for you.
If you enjoyed this episode check out "Geospatial Side hustles"
https://mapscaping.com/podcast/geospatial-side-hustles/
Please support this podcast on Patreon!
https://www.patreon.com/MapScaping?
Join the mailing list!
https://mapscaping.com/podcasts/