[APICORE-980] Propagate gdal error messages instead of completely obfuscating them

[daniel-cohen]

This is an initial draft PR to discuss the change.
We initially did not want to rely on string messages to determine 404 errors, But I think this will be an improvement for API consumer . This will allow us to determine if the file is actually missing in s3 and return a 404.

if we want to remain 100% backwards compatible, we can create a new function and leave the current gdal.Open as is.
The new function can be used in sourcephotos:

Some example outputs:

For 404:

New error:: "Error: HTTP response code: 404. dataset '/vsicurl/http://127.0.0.1:40967/%2Fphotos-proxy%2Fv1%2Fhv-readonly01-ap-southeast-2%2FHyperStorage%2FReadOnly%2FProjects%2FMaps%2FNSW.SydneyTest.421%2C%20AU%2FSurveys%2F2022-10-29%20%28Oct%202022%29%20080222%2FPhotos%2F2022-10-29%200802%2FLine-003%2FTilt-2%2F2233_511749.150232.tif' 

Old error: "Error: dataset '/vsicurl/http://127.0.0.1:40967/%2Fphotos-proxy%2Fv1%2Fhv-readonly01-ap-southeast-2%2FHyperStorage%2FReadOnly%2FProjects%2FMaps%2FNSW.SydneyTest.421%2C%20AU%2FSurveys%2F2022-10-29%20%28Oct%202022%29%20080222%2FPhotos%2F2022-10-29%200802%2FLine-003%2FTilt-2%2F2233_511749.150232.tif' open error"

---

Couldn't resolve host:

New error: "Error: CURL error: Could not resolve host: http. dataset '/vsicurl/http://http://127.0.0.1:40523/%2Fphotos-proxy%2Fv1%2Fhv-readonly01-ap-southeast-2%2FHyperStorage%2FReadOnly%2FProjects%2FMaps%2FNSW.SydneyTest.421%2C%20AU%2FSurveys%2F2022-10-29%20%28Oct%202022%29%20080222%2FPhotos%2F2022-10-29%200802%2FLine-003%2FTilt-2%2F2233_511749.150232.tif' open error"

Old error: "Error: dataset '/vsicurl/http://http://127.0.0.1:40523/%2Fphotos-proxy%2Fv1%2Fhv-readonly01-ap-southeast-2%2FHyperStorage%2FReadOnly%2FProjects%2FMaps%2FNSW.SydneyTest.421%2C%20AU%2FSurveys%2F2022-10-29%20%28Oct%202022%29%20080222%2FPhotos%2F2022-10-29%200802%2FLine-003%2FTilt-2%2F2233_511749.150232.tif' open error"

---

I was messing around and added "xx" to the URL

New error:  "Error: HTTP response code: 0. dataset '/vsicurl/http://XXhttp://127.0.0.1:45115/photos-proxy/v1/hv-readonly01-ap-southeast-2/HyperStorage/ReadOnly/Projects/Maps/NSW.SydneyTest.421, AU/Surveys/2022-10-29 (Oct 2022) 080222/Photos/2022-10-29 0802/Line-003/Tilt-2/2233_511749.150232.tif' open error"

Old error: "Error: dataset '/vsicurl/http://XXhttp://127.0.0.1:45115/photos-proxy/v1/hv-readonly01-ap-southeast-2/HyperStorage/ReadOnly/Projects/Maps/NSW.SydneyTest.421, AU/Surveys/2022-10-29 (Oct 2022) 080222/Photos/2022-10-29 0802/Line-003/Tilt-2/2233_511749.150232.tif' open error"

This was in the logs when I enabled full verbose GDAL logs:

ERROR 11: HTTP response code: 0
ERROR 4: `/vsicurl/http://XXhttp://127.0.0.1:45115/photos-proxy/v1/hv-readonly01-ap-southeast-2/HyperStorage/ReadOnly/Projects/Maps/NSW.SydneyTest.421, AU/Surveys/2022-10-29 (Oct 2022) 080222/Photos/2022-10-29 0802/Line-003/

[daniel-cohen]

Quick test to simulate the sourcephotos failures:

package main

import (
	"fmt"
	"net/http"
	"net/http/httptest"
	"net/url"

	"github.com/nearmap/gdal"
)

func main() {

	//test server:
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		http.Error(w, "404 page not found", http.StatusNotFound)
		return
	}))
	defer server.Close()

	// GDAL's usage of libcurl doesn't like unescaped paths.
	path := "/photos-proxy/v1/hv-readonly01-ap-southeast-2/HyperStorage/ReadOnly/Projects/Maps/NSW.SydneyTest.421, AU/Surveys/2022-10-29 (Oct 2022) 080222/Photos/2022-10-29 0802/Line-003/Tilt-2/2233_511749.150232.tif"
	path = url.PathEscape(path)

	sampleRequestURL := fmt.Sprintf("/vsicurl/%s/%s", server.URL, path)

	fmt.Println("New error:")
	ds, err := gdal.OpenWithError(sampleRequestURL, gdal.ReadOnly)
	if err != nil {
		fmt.Println("---------------------------------------------")
		fmt.Printf("GDAL ERROR: %v\n", err)
		fmt.Println("---------------------------------------------")
	} else {
		fmt.Println("+++++++++++++++++++++++++++++++++++++++++++++")
		fmt.Println("no error")
		fmt.Printf("ds=%v\n", ds)
		fmt.Println("+++++++++++++++++++++++++++++++++++++++++++++")
	}

	// old error:

	fmt.Println("old error:")
	ds, err = gdal.Open(sampleRequestURL, gdal.ReadOnly)
	if err != nil {
		fmt.Println("---------------------------------------------")
		fmt.Printf("GDAL ERROR: %v\n", err)
		fmt.Println("---------------------------------------------")
	} else {
		fmt.Println("+++++++++++++++++++++++++++++++++++++++++++++")
		fmt.Println("no error")
		fmt.Printf("ds=%v\n", ds)
		fmt.Println("+++++++++++++++++++++++++++++++++++++++++++++")
	}

}