Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
2ab61a3
messing around with water use
ldecicco-USGS Jan 21, 2026
cea389a
cleanup with develop
ldecicco-USGS Jan 21, 2026
e70ff54
Just use csv, it's a little faster
ldecicco-USGS Jan 22, 2026
66800ca
upstream pull
ldecicco-USGS Feb 18, 2026
758533b
getting ready for NGWMN
ldecicco-USGS Mar 5, 2026
98a441d
gah
ldecicco-USGS Mar 9, 2026
3379d61
get all functions ready
ldecicco-USGS Mar 9, 2026
9ee4770
Get upstream data
ldecicco-USGS Mar 11, 2026
7d51660
from upstream
ldecicco-USGS Apr 20, 2026
72b2aa2
Updates from develop
ldecicco-USGS Apr 29, 2026
e32db79
Merge branch 'develop' of https://code.usgs.gov/water/dataRetrieval i…
ldecicco-USGS May 7, 2026
f45e702
upstream updates
ldecicco-USGS May 19, 2026
8f3ac58
Merge branch 'develop' of https://code.usgs.gov/water/dataRetrieval i…
ldecicco-USGS May 19, 2026
c881609
Merge branch 'main' of github.com:DOI-USGS/dataRetrieval into ngwmn
ldecicco-USGS Jun 2, 2026
c2947ba
some updates from API
ldecicco-USGS Jun 2, 2026
a125abf
updates from APIs
ldecicco-USGS Jun 2, 2026
977d870
rebuilding docs
ldecicco-USGS Jun 2, 2026
aba54d7
remove old code
ldecicco-USGS Jun 2, 2026
c2caa6d
Add more time columns
ldecicco-USGS Jun 4, 2026
39f9c26
Merge pull request #903 from ldecicco-USGS/develop
ldecicco-USGS Jun 4, 2026
424b6f9
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into ngwmn
ldecicco-USGS Jun 11, 2026
f06fcf5
remove old examples, add new ones
ldecicco-USGS Jun 11, 2026
9c15b1c
Upstream pull
ldecicco-USGS Jun 11, 2026
b85db63
updated docs
ldecicco-USGS Jun 11, 2026
95e4113
rerun docs
ldecicco-USGS Jun 15, 2026
c8c6dbd
Update R/get_ogc_documentation.R
ldecicco-USGS Jun 16, 2026
4758026
Add check for required arguments
ldecicco-USGS Jun 16, 2026
77df23c
Merge branch 'ngwmn' of github.com:ldecicco-USGS/dataRetrieval into n…
ldecicco-USGS Jun 16, 2026
7bbf399
add metadata output to read_waterdata_ratings output
mconlon91 Jun 16, 2026
d818d77
remove stringsAsFactors
mconlon91 Jun 18, 2026
11cd293
Merge branch 'ratings_metadata' into 'develop'
ldecicco-USGS Jun 22, 2026
cacc4ed
fix na bug
ldecicco-USGS Jun 23, 2026
834741b
Merge branch 'develop' of https://code.usgs.gov/water/dataRetrieval i…
ldecicco-USGS Jun 23, 2026
ef7756a
building the docs
ldecicco-USGS Jun 23, 2026
7f65eb9
Adding read_ngwmn function
ldecicco-USGS Jun 23, 2026
6c6a2f7
Add some tests
ldecicco-USGS Jun 23, 2026
0e79ee8
add more tests
ldecicco-USGS Jun 23, 2026
aa0ee50
Merge pull request #904 from ldecicco-USGS/ngwmn
ldecicco-USGS Jun 23, 2026
682bbe0
update wateruse
ldecicco-USGS Jun 23, 2026
714f6ca
upstream pull
ldecicco-USGS Jun 23, 2026
d5a4499
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into develop
ldecicco-USGS Jun 23, 2026
cc009c1
add tests
ldecicco-USGS Jun 23, 2026
4c1d0c9
rename
ldecicco-USGS Jun 23, 2026
f1d8510
rename file
ldecicco-USGS Jun 23, 2026
cb6e1bd
remove functions that don't work
ldecicco-USGS Jun 23, 2026
ddaca79
Cleanup pkgdown
ldecicco-USGS Jun 23, 2026
67191be
Merge pull request #905 from ldecicco-USGS/wateruse
ldecicco-USGS Jun 24, 2026
c7fe179
Allow geometry in continuous
ldecicco-USGS Jun 24, 2026
bbfcca7
fix note
ldecicco-USGS Jun 24, 2026
23e4b46
cleanup metadata list
ldecicco-USGS Jul 7, 2026
e94f179
latest field
ldecicco-USGS Jul 7, 2026
7e620d8
from upstream
ldecicco-USGS Jul 7, 2026
fb8e365
from upstream
ldecicco-USGS Jul 7, 2026
3f01762
upstream pull
ldecicco-USGS Jul 7, 2026
238ec0a
Merge pull request #911 from thodson-usgs/fix/ogc-properties-id-400
ldecicco-USGS Jul 7, 2026
5ac5c76
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into late…
ldecicco-USGS Jul 7, 2026
dca11c0
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into upda…
ldecicco-USGS Jul 7, 2026
51792cf
First pass at latest_field
ldecicco-USGS Jul 7, 2026
ffc5adb
adding non-returned query arguments
ldecicco-USGS Jul 8, 2026
cce5080
add service
ldecicco-USGS Jul 8, 2026
eb88086
change name
ldecicco-USGS Jul 8, 2026
616e04d
Merge pull request #912 from ldecicco-USGS/latest_field
ldecicco-USGS Jul 8, 2026
93383c5
more functions with queries
ldecicco-USGS Jul 8, 2026
ef545cb
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into upda…
ldecicco-USGS Jul 8, 2026
82540e9
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into develop
ldecicco-USGS Jul 8, 2026
4885457
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into extr…
ldecicco-USGS Jul 8, 2026
1cb928e
more functions with extra queries
ldecicco-USGS Jul 8, 2026
0429038
pkgdown
ldecicco-USGS Jul 8, 2026
3d98bba
Merge branch 'develop' of https://code.usgs.gov/water/dataRetrieval i…
ldecicco-USGS Jul 8, 2026
cf289fb
Merge pull request #909 from ldecicco-USGS/updates_to_ratings_meta
ldecicco-USGS Jul 8, 2026
c3a77cb
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into extr…
ldecicco-USGS Jul 8, 2026
600ef57
adding q
ldecicco-USGS Jul 8, 2026
717d7ea
bring these back someday
ldecicco-USGS Jul 8, 2026
c0577a1
update docs
ldecicco-USGS Jul 8, 2026
fd52921
Change to dots
ldecicco-USGS Jul 9, 2026
be7fa99
dots
ldecicco-USGS Jul 9, 2026
79aec53
Specific NGWMN dot
ldecicco-USGS Jul 9, 2026
3f87265
simplify to just dots
ldecicco-USGS Jul 9, 2026
3444e4e
Added news
ldecicco-USGS Jul 9, 2026
c16f218
sp
ldecicco-USGS Jul 9, 2026
5b178bd
cleanup arg
ldecicco-USGS Jul 9, 2026
6d0ea74
get NEWS added to pkgdown
ldecicco-USGS Jul 13, 2026
d12f9c3
with logo
ldecicco-USGS Jul 13, 2026
358148a
Add subtitles
ldecicco-USGS Jul 16, 2026
540f272
?
ldecicco-USGS Jul 16, 2026
2629cee
no prompt
ldecicco-USGS Jul 16, 2026
efd745e
I think this will be better because local installations will work wit…
ldecicco-USGS Jul 16, 2026
310d3a2
Repoint CQL to OGC sources
mikemahoney218-usgs Jul 21, 2026
a40a307
Merge pull request #914 from mikemahoney218-usgs/patch-1
ldecicco-USGS Jul 21, 2026
221f297
Win+Ctrl+L seems like a better way to do transcripts if needed
ldecicco-USGS Jul 27, 2026
6832714
response to review
ldecicco-USGS Aug 3, 2026
d7a9c01
missed a few commits
ldecicco-USGS Aug 3, 2026
bf1de24
Moving this to next release
ldecicco-USGS Aug 4, 2026
73bd6bb
Merge pull request #913 from ldecicco-USGS/extra_query_params
ldecicco-USGS Aug 4, 2026
f71ceb6
update News
ldecicco-USGS Aug 4, 2026
097903a
Merge pull request #915 from ldecicco-USGS/develop
ldecicco-USGS Aug 4, 2026
6a59ab6
Getting CRAN ready
ldecicco-USGS Aug 4, 2026
5b30b41
rename
ldecicco-USGS Aug 4, 2026
559a019
Merge pull request #916 from ldecicco-USGS/develop
ldecicco-USGS Aug 4, 2026
a9a8d00
fix news
ldecicco-USGS Aug 4, 2026
ad3a40a
Merge pull request #918 from ldecicco-USGS/develop
ldecicco-USGS Aug 4, 2026
638351a
use time_periods
ldecicco-USGS Aug 4, 2026
d1062d8
Fix status
ldecicco-USGS Aug 4, 2026
dcaa3da
Merge pull request #919 from ldecicco-USGS/develop
ldecicco-USGS Aug 4, 2026
cd0e2d4
still deprecated
ldecicco-USGS Aug 4, 2026
8a6307b
Update R/get_monitoring_location_arguments.R
ldecicco-USGS Aug 4, 2026
5553626
Update R/get_monitoring_location_arguments.R
ldecicco-USGS Aug 4, 2026
5b84e35
Merge branch 'develop' of github.com:DOI-USGS/dataRetrieval into develop
ldecicco-USGS Aug 4, 2026
cb9b6ad
cleanup message
ldecicco-USGS Aug 5, 2026
b46879f
Merge pull request #920 from ldecicco-USGS/develop
ldecicco-USGS Aug 5, 2026
2208c4d
spelling
ldecicco-USGS Aug 5, 2026
1c0b6c1
Merge pull request #921 from ldecicco-USGS/develop
ldecicco-USGS Aug 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/pkgdown.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,8 @@ jobs:
- name: Build site
run: |
pkgdown::build_site(override = list(destination = "public")) |
file.copy(from = "./public/articles/logo.png",to = "./public/reference/logo.png")
file.copy(from = "./public/articles/logo.png",to = "./public/reference/logo.png") |
file.copy(from = "./public/articles/logo.png", to = "./public/news/logo.png")
shell: Rscript {0}
- name: Set up Quarto
uses: quarto-dev/quarto-actions/setup@v2
Expand Down
3 changes: 2 additions & 1 deletion .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,8 @@ pages:
- |
Rscript -e '
pkgdown::build_site(override = list(destination = "public"))
file.copy(from = "./public/articles/logo.png", to = "./public/reference/logo.png")'
file.copy(from = "./public/articles/logo.png", to = "./public/reference/logo.png")
file.copy(from = "./public/articles/logo.png", to = "./public/news/logo.png")'
- quarto render
artifacts:
paths:
Expand Down
7 changes: 5 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: dataRetrieval
Type: Package
Title: Retrieval Functions for USGS and EPA Hydrology and Water Quality Data
Version: 2.7.25
Version: 2.7.26
Authors@R: c(
person("Laura", "DeCicco", role = c("aut","cre"),
email = "ldecicco@usgs.gov",
Expand Down Expand Up @@ -34,7 +34,10 @@ Authors@R: c(
comment=c(ORCID = "0000-0001-5396-1583")),
person("Michael", "Mahoney", role="ctb",
email = "mjmahoney@usgs.gov",
comment=c(ORCID = "0000-0003-2402-304X"))
comment=c(ORCID = "0000-0003-2402-304X")),
person("Matthew", "Conlon", role="ctb",
email = "mconlon@usgs.gov",
comment=c(ORCID = "0000-0001-8266-9610"))
)
Description: Collection of functions to help retrieve U.S. Geological Survey
and U.S. Environmental Protection Agency water quality and
Expand Down
11 changes: 8 additions & 3 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export(create_NWIS_bib)
export(create_WQP_bib)
export(findNLDI)
export(getWebServiceData)
export(get_monitoring_location_arguments)
export(get_nldi_sources)
export(get_ogc_params)
export(importNGWMN)
Expand All @@ -27,9 +28,6 @@ export(pCodeToName)
export(parameterCdFile)
export(parse_WQP)
export(pcode_to_name)
export(readNGWMNdata)
export(readNGWMNlevels)
export(readNGWMNsites)
export(readNWISdata)
export(readNWISdv)
export(readNWISpCode)
Expand All @@ -43,6 +41,12 @@ export(readWQPdata)
export(readWQPqw)
export(readWQPsummary)
export(read_USGS_samples)
export(read_ngwmn)
export(read_ngwmn_lithology)
export(read_ngwmn_providers)
export(read_ngwmn_sites)
export(read_ngwmn_water_level)
export(read_ngwmn_well_construction)
export(read_waterdata)
export(read_waterdata_channel)
export(read_waterdata_combined_meta)
Expand All @@ -52,6 +56,7 @@ export(read_waterdata_field_measurements)
export(read_waterdata_field_meta)
export(read_waterdata_latest_continuous)
export(read_waterdata_latest_daily)
export(read_waterdata_latest_field_measurements)
export(read_waterdata_metadata)
export(read_waterdata_monitoring_location)
export(read_waterdata_parameter_codes)
Expand Down
26 changes: 18 additions & 8 deletions NEWS → NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
dataRetrieval 2.7.26
===================
* Added suite of read_ngwmn to access new National Groundwater Monitoring
Network API.
* Created get_monitoring_location_arguments to help users understand
what new optional arguments are available in many of the read_waterdata functions.
* Added ability to use `...` in the Water Data functions for: daily, latest-continuous, field-measurements, latest-daily, latest-field-measurements, continuous, and peaks. The "..." argument can now accept any parameter that is output from the make_monitoring_location_arguments function.
* Added "q" argument to read_waterdata_monitoring_location. Full-text search across the most relevant text fields for this collection (e.g. site name, identifier, number, state, county, and site type).
* Added "skipGeometry" argument to read_waterdata_continuous. Previously geometry wasn't included with continuous data, but now is supported.

dataRetrieval 2.7.25
===================
* Added read_waterdata_ratings to access USGS rating curves with
Expand Down Expand Up @@ -32,27 +42,27 @@ return full period of record.

dataRetrieval 2.7.23
===================
* Added data.table to Imports

* Added read_waterdata_stats_por and read_waterdata_stats_daterange to access USGS daily data statistics.
* Updated vignettes, examples, and README to reflect new stats functions.
* Added tests for read_waterdata_stats functions
* Added read_waterdata_field_meta, read_waterdata_combine_meta, and read_waterdata_channel
* Added read_ngwmn_water_levels, read_ngwmn_sites, read_ngwmn_providers, read_ngwmn_well_construction, read_ngwmn_lithology
* Removed readNWISgwl, readNWISmeas, and readNGWMN as services have been turned off
* Added data.table to Imports
* Updated next_req_url to allow paging through /statistics API output
* Added deprecation message to readNWISstat function
* Specify "UTC" attribute for returned time
* Improved time documentation for waterdata functions
* Specify "UTC" attribute for returned time for waterdata functinos
* Added options("dataRetrieval.attach_request" = TRUE) as default to attach the
request object as an attribute to the returned data frames. Setting
options("dataRetrieval.attach_request" = FALSE) will return a data frame without
the request attribute
* Improved time documentation
* Improved error handling in WQP functions
* Added read_waterdata_field_meta, read_waterdata_combine_meta,
and read_waterdata_channel
* Removed readNWISgwl and readNWISmeas as services have been turned off
* Updated CQL2 templates to allow HUC queries specifically to use a wildcard
to get multiple inclusive HUCs.
* Add deprecation message to readNGWMN functions. read_ngwmn will be coming soon.



dataRetrieval 2.7.22
===================
* Added read_waterdata_latest_daily to access latest daily USGS water data.
Expand Down
6 changes: 5 additions & 1 deletion R/AAA.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@ pkg.env <- new.env()
.onLoad <- function(libname, pkgname) {
suppressMessages(setAccess("public"))
pkg.env$local_sf <- requireNamespace("sf", quietly = TRUE)
options("dataRetrieval.nldi_base" = "https://api.water.usgs.gov/nldi/linked-data/")
options(
"dataRetrieval.nldi_base" = "https://api.water.usgs.gov/nldi/linked-data/"
)
options("dataRetrieval.api_version" = "v0")
options("dataRetrieval.api_version_stat" = "v0")
options("dataRetrieval.attach_request" = TRUE)
Expand All @@ -21,6 +23,7 @@ pkg.env <- new.env()
"latest-continuous",
"field-measurements",
"latest-daily",
"latest-field-measurements",
"continuous",
"field-measurements-metadata",
"combined-metadata",
Expand Down Expand Up @@ -78,6 +81,7 @@ pkg.env <- new.env()
"time_zone_code"
)

num_cols <- c()
pkg.env$api_endpoints <- services
pkg.env$metadata <- collections
}
Expand Down
34 changes: 20 additions & 14 deletions R/construct_api_requests.R
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ construct_api_requests <- function(
output_id,
...,
bbox = NA,
base = "OGC",
convertType = getOption("dataRetrieval.convertType"),
no_paging = getOption("dataRetrieval.no_paging"),
chunk_size = getOption("dataRetrieval.site_chunk_size_data"),
Expand Down Expand Up @@ -92,13 +93,7 @@ construct_api_requests <- function(
}

single_params <- c(
"datetime",
"last_modified",
"begin",
"end",
"time",
"begin_utc",
"end_utc",
time_periods,
"limit",
"skipGeometry"
)
Expand Down Expand Up @@ -159,7 +154,7 @@ construct_api_requests <- function(

format_type <- ifelse(isTRUE(no_paging), "csv", "json")

baseURL <- setup_api(service, format = format_type)
baseURL <- setup_api(service, format = format_type, base = base)
baseURL <- explode_query(baseURL, POST = FALSE, get_list, multi = "comma")

if (all(!is.na(bbox))) {
Expand Down Expand Up @@ -235,16 +230,25 @@ construct_api_requests <- function(
#' Setup the request for the OGC API requests
#'
#' @noRd
#' @param base Character OGC or NGWMN
#' @return httr2 request
#' @examplesIf is_dataRetrieval_user()
#'
#' \donttest{
#' request <- dataRetrieval:::base_url()
#' request
#' }
base_url <- function() {
httr2::request("https://api.waterdata.usgs.gov/ogcapi/") |>
httr2::req_url_path_append(getOption("dataRetrieval.api_version"))
base_url <- function(base = "OGC") {
match.arg(base, c("OGC", "NGWMN"))

if (base == "OGC") {
baseURL <- httr2::request("https://api.waterdata.usgs.gov/ogcapi/") |>
httr2::req_url_path_append(getOption("dataRetrieval.api_version"))
} else {
baseURL <- httr2::request(
"https://api.waterdata.usgs.gov/ngwmn/ogcapi/"
)
}
}

#' Setup the request for a particular endpoint collection
Expand All @@ -257,8 +261,8 @@ base_url <- function() {
#' request <- dataRetrieval:::setup_api("daily")
#' request
#' }
setup_api <- function(service, format = "json") {
baseURL <- base_url() |>
setup_api <- function(service, format = "json", base = "OGC") {
baseURL <- base_url(base) |>
httr2::req_url_path_append("collections") |>
httr2::req_url_path_append(service, "items") |>
basic_request(format = format)
Expand Down Expand Up @@ -654,5 +658,7 @@ time_periods <- c(
"begin",
"end",
"begin_utc",
"end_utc"
"end_utc",
"revision_created",
"revision_modified"
)
5 changes: 3 additions & 2 deletions R/dataRetrieval-package.R
Original file line number Diff line number Diff line change
Expand Up @@ -208,14 +208,15 @@ NULL
# "field-measurements", "latest-daily",
# "continuous", "field-measurements-metadata",
# "combined-metadata", "channel-measurements", "peaks")
#
#
# property_list <- list()
# for(service in services){
# property_list[[service]] <- dataRetrieval:::get_properties_for_docs(service)
# }
#
# num_cols <- c("value", "contributing_drainage_area", "drainage_area",
# "altitude_accuracy", "well_construction_depth",
# #"altitude_accuracy",
# "well_construction_depth",
# "hole_construction_depth", "channel_flow", "channel_width",
# "channel_area", "channel_velocity", "channel_location_distance")
#
Expand Down
14 changes: 9 additions & 5 deletions R/deal_with_empty.R
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#' as "daily", "monitoring-locations", "time-series-metadata"
#' @param skipGeometry A logical for whether to return geometry
#' @param convertType A logical for whether to convert value to numeric
#' @param base character, either "OGC" or "NGWMN" depending on data.
#'
#' @return data.frame
#' @noRd
Expand All @@ -25,20 +26,23 @@ deal_with_empty <- function(
service,
skipGeometry,
convertType,
no_paging = FALSE
no_paging = FALSE,
base = "OGC"
) {
if (nrow(return_list) == 0) {
if (all(is.na(properties))) {
schema <- check_OGC_requests(endpoint = service, type = "schema")
schema <- check_OGC_requests(
endpoint = service,
type = "schema",
base = base
)
properties <- names(schema$properties)
}
return_list <- data.frame(matrix(nrow = 0, ncol = length(properties)))
return_list <- lapply(return_list, as.character)
names(return_list) <- properties

single_params <- c("datetime", "last_modified", "begin", "end", "time")

for (i in single_params) {
for (i in time_periods) {
if (i %in% names(return_list)) {
return_list[[i]] <- as.POSIXct(as.character(), origin = "1970-01-01")
}
Expand Down
Loading
Loading