# Raster streaming best practices Streaming Cloud Optimized GeoTIFFs (COGs) into Felt allows you to work with massive raster datasets without uploading them directly to Felt's servers. By creating a Virtual Raster (VRT) that references COGs stored in your cloud storage, you can visualize terabyte-scale datasets efficiently. However, performance depends on several key factors that you should optimize for the best experience. {% hint style="success" %} This feature is only available to customers on the [Enterprise plan](https://felt.com/pricing). To upgrade, [contact sales](https://felt.com/sales). {% endhint %} ## How streaming performance works When you stream rasters into Felt, the platform fetches data on-demand from your cloud storage to generate map tiles. Performance is determined by three main factors: 1. **Source GeoTIFF format and configuration:** how your individual COG files are structured 2. **Storage location**: where your files are hosted and network latency 3. **Geographical scope of source files:** the geographic area each file covers relative to the map tiles being requested ### Best Practices Check out [#optimization-strategies](#optimization-strategies "mention") for in-depth details. * Use properly formatted Cloud Optimized GeoTIFFs with 512×512 internal blocks * Create larger-footprint files (covering more geographic area) for better low-zoom performance * Host files in AWS S3, preferably in the same region as Felt * Set visibility limits to hide layers at zoom levels where they perform poorly * Test with smaller datasets before deploying full-scale data * Include pre-calculated overviews in your COGs **Avoid:** * Using many small-footprint GeoTIFF files (like 1° × 1°) if you need good low-zoom performance * Hosting files in storage with high latency or cross-region access * Attempting to render global datasets at very low zoom levels without optimization * Skipping internal tiling or overviews in your COG files ### Understanding zoom level limitations The performance of streamed rasters varies significantly by zoom level. As you zoom out to view larger areas, Felt must fetch overview data from more individual GeoTIFF files to construct each map tile. #### Example Assume we have a dataset with 1° × 1° GeoTIFF files, here's approximately how many files need to be accessed per tile at each zoom level: * **Zoom 10-8**: 1-2 files (excellent performance) * **Zoom 7**: 6 files (good performance) * **Zoom 6**: 24 files (acceptable performance) * **Zoom 5**: 90 files (performance degradation begins) * **Zoom 4**: 333 files (slow performance) * **Zoom 3**: 1,150+ files (very slow, likely timeouts) * **Zoom 2 and below**: 5,000+ files (will not work) If the system cannot fetch and process all required data within this window, the tile request will fail. ## Optimization strategies ### 1. Optimize GeoTIFF file configuration Ensure your COG files are properly formatted: Recommended settings: * **Internal tiling**: Use 512×512 pixel blocks (optimal for web tile generation) * **Compression**: Use LZW or DEFLATE compression to reduce file size * **Overviews**: Include multiple overview levels (pyramids) in each GeoTIFF * **File format**: Verify files are true Cloud Optimized GeoTIFFs You can create properly formatted COGs using GDAL, for example: ```bash gdal_translate input.tif output.tif \ -co TILED=YES \ -co BLOCKXSIZE=512 \ -co BLOCKYSIZE=512 \ -co COMPRESS=DEFLATE \ -co COPY_SRC_OVERVIEWS=YES \ -co BIGTIFF=IF_SAFER ``` ### 2. Increase individual GeoTIFF footprints The most effective way to improve lower-zoom performance is to create larger-area COG files. Instead of many small files (like 1° × 1°), consider creating files that cover larger geographic extents (like 5° × 5° or 10° × 10°). The benefits are: * Fewer individual files need to be accessed per tile request * Larger files have additional overview levels that can be used at lower zoom levels * Optimizes the number of network requests required There are some trade-offs to consider * Larger individual files take longer to process initially * This approach may be less convenient for regional data access * Requires regenerating your dataset ### 3. Set appropriate visibility limits If your dataset performs poorly at lower zoom levels, prevent rendering issues by setting visibility constraints. There are two ways you can do this in Felt: [Layer-level limits](https://help.felt.com/layers/styling/vector-layers#zoom-based-styling) 1. Select your raster layer 2. Open layer settings 3. Set "Limit visibility" to only show at appropriate zoom levels (e.g., zoom 6 and higher) [Map constraints](https://help.felt.com/dashboards-and-apps/map-settings#map-constraints) 1. Open map settings 2. Set map constraints to prevent zooming out beyond viable zoom levels #### 4. Consider VRT statistics If you're using a VRT file, pre-calculate summary statistics to improve initial rendering. You can run use GDAL to do this: ```bash gdalinfo -stats your_file.vrt ``` Without pre-calculated stats, Felt may need to scan the entire dataset to determine appropriate color ramps and styling, which can be slow for large datasets. ## Testing When working with large streaming datasets, follow this testing approach: 1. **Start small**: Test with a subset of your data (e.g., 10-20 GB) first 2. **Test at different zoom levels**: Navigate to various zoom levels and note where performance degrades 3. **Monitor tile load times**: Watch the network requests in your browser's developer tools 4. **Iterate on optimization**: Adjust your COG configuration based on performance results 5. **Scale up gradually**: Once optimized, test with progressively larger datasets ## Troubleshooting If you're working with a large streaming raster dataset and encountering performance issues, reach out to [Felt's support team](mailto:support@felt.com). They can help analyze your specific use case and provide guidance on optimization strategies. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://help.felt.com/data-sources/raster-infrastructure/raster-streaming-best-practices.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.