Guide to raster streaming

Best practices for streaming raster data into Felt.

Streaming Cloud Optimized GeoTIFFs (COGs) into Felt allows you to work with massive raster datasets without uploading them directly to Felt's servers. By creating a Virtual Raster (VRT) that references COGs stored in your cloud storage, you can visualize terabyte-scale datasets efficiently. However, performance depends on several key factors that you should optimize for the best experience.

This feature is only available to customers on the Enterprise plan. To upgrade, contact sales.

How streaming performance works

When you stream rasters into Felt, the platform fetches data on-demand from your cloud storage to generate map tiles. Performance is determined by three main factors:

1. Source GeoTIFF format and configuration: how your individual COG files are structured

2. Storage location: where your files are hosted and network latency

3. Geographical scope of source files: the geographic area each file covers relative to the map tiles being requested

Best Practices

Check out Optimization strategies for in-depth details.

• Use properly formatted Cloud Optimized GeoTIFFs with 512×512 internal blocks

• Create larger-footprint files (covering more geographic area) for better low-zoom performance

• Host files in AWS S3, preferably in the same region as Felt

• Set visibility limits to hide layers at zoom levels where they perform poorly

• Test with smaller datasets before deploying full-scale data

• Include pre-calculated overviews in your COGs

Avoid:

• Using many small-footprint GeoTIFF files (like 1° × 1°) if you need good low-zoom performance

• Hosting files in storage with high latency or cross-region access

• Attempting to render global datasets at very low zoom levels without optimization

• Skipping internal tiling or overviews in your COG files

Understanding zoom level limitations

The performance of streamed rasters varies significantly by zoom level. As you zoom out to view larger areas, Felt must fetch overview data from more individual GeoTIFF files to construct each map tile.

Example

Assume we have a dataset with 1° × 1° GeoTIFF files, here's approximately how many files need to be accessed per tile at each zoom level:

• Zoom 10-8: 1-2 files (excellent performance)

• Zoom 7: 6 files (good performance)

• Zoom 6: 24 files (acceptable performance)

• Zoom 5: 90 files (performance degradation begins)

• Zoom 4: 333 files (slow performance)

• Zoom 3: 1,150+ files (very slow, likely timeouts)

• Zoom 2 and below: 5,000+ files (will not work)

If the system cannot fetch and process all required data within this window, the tile request will fail.

Optimization strategies

1. Optimize GeoTIFF file configuration

Ensure your COG files are properly formatted:

Recommended settings:

• Internal tiling: Use 512×512 pixel blocks (optimal for web tile generation)

• Compression: Use LZW or DEFLATE compression to reduce file size

• Overviews: Include multiple overview levels (pyramids) in each GeoTIFF

• File format: Verify files are true Cloud Optimized GeoTIFFs

You can create properly formatted COGs using GDAL, for example:

gdal_translate input.tif output.tif \
  -co TILED=YES \
  -co BLOCKXSIZE=512 \
  -co BLOCKYSIZE=512 \
  -co COMPRESS=DEFLATE \
  -co COPY_SRC_OVERVIEWS=YES \
  -co BIGTIFF=IF_SAFER

2. Increase individual GeoTIFF footprints

The most effective way to improve lower-zoom performance is to create larger-area COG files. Instead of many small files (like 1° × 1°), consider creating files that cover larger geographic extents (like 5° × 5° or 10° × 10°). The benefits are:

• Fewer individual files need to be accessed per tile request

• Larger files have additional overview levels that can be used at lower zoom levels

• Optimizes the number of network requests required

There are some trade-offs to consider

• Larger individual files take longer to process initially

• This approach may be less convenient for regional data access

• Requires regenerating your dataset

3. Set appropriate visibility limits

If your dataset performs poorly at lower zoom levels, prevent rendering issues by setting visibility constraints. There are two ways you can do this in Felt:

Layer-level limits

1. Select your raster layer

2. Open layer settings

3. Set "Limit visibility" to only show at appropriate zoom levels (e.g., zoom 6 and higher)

Map constraints

1. Open map settings

2. Set map constraints to prevent zooming out beyond viable zoom levels

4. Consider VRT statistics

If you're using a VRT file, pre-calculate summary statistics to improve initial rendering. You can run use GDAL to do this:

gdalinfo -stats your_file.vrt

Without pre-calculated stats, Felt may need to scan the entire dataset to determine appropriate color ramps and styling, which can be slow for large datasets.

Testing

When working with large streaming datasets, follow this testing approach:

1. Start small: Test with a subset of your data (e.g., 10-20 GB) first

2. Test at different zoom levels: Navigate to various zoom levels and note where performance degrades

3. Monitor tile load times: Watch the network requests in your browser's developer tools

4. Iterate on optimization: Adjust your COG configuration based on performance results

5. Scale up gradually: Once optimized, test with progressively larger datasets

Troubleshooting

If you're working with a large streaming raster dataset and encountering performance issues, reach out to Felt's support team. They can help analyze your specific use case and provide guidance on optimization strategies.