monitor_ggDailyBarplot_archival()

[jonathancallahan]

We need a new function to create "daily barplots" for archival data.

We will want to improve this plot iteratively, with feedback from the AirFire team. The starting point is the monitor_ggDailyBarplot() function. The new, "Archival" version of the function will have the following changes:

• add monitorID in parens after/under site name
• include the year underneath the x-axis
• x-axis tic labels so that small, clear labels are present for any time range form 1 -12 months
• at larger time ranges, switch to bars with transparent/missing borders so that the plot isn't overwhelmed by the black borders drawn around each bar
• explore why the plot is often drawn with a "missing data" bar at the beginning/end of the time period. (This could be an issue where data are trimmed using UTC dates and then displayed in local time or vice versa thus resulting in some days with not enough points.)

[tabrasel]

monitor_ggDailyBarplot() often includes gray bars at the beginning end because it internally calls stat_dailyAQCategory() which sets missingDataBar = TRUE by default. Down the line, this "extends" the date range to its full extent by adding data entries to the beginning and end with empty values:

StatDailyAQILevel <- ggproto(...) {

  ...

  # Add missing data bars
  if (missingDataBar) {

    # Extend data to full extent
    max_x <- max(data$x)
    while (max_x < scales$x$get_limits()[2]) {
      max_x <- max_x + 86400
      data <- rbind(data, tibble(
        x = max_x,
        y = NA,
        aqi = NA,
        colour = NA,
        fill = NA
      ))
    }

    min_x <- min(data$x)
    while (min_x > scales$x$get_limits()[2]) {
      min_x <- min_x - 86400
      data <- rbind(data, tibble(
        x = min_x,
        y = NA,
        aqi = NA,
        colour = NA,
        fill = NA
      ))
    }

    # Add gray bars
    for (missingRow in which(is.na(data$y))) {
      data[missingRow, "y"] <- scales$y$get_limits()[2]
    }
  }

  ...
}

We can avoid drawing these empty end bars by setting missingDataBar = TRUE in the call to stat_dailyAQCategory(), but this means that any days between with missing values also won't display a gray bar.

[jonathancallahan]

We DO want missing data bars to be shown in gray. We DON'T want to include an extra day at the end of the requested time period.

This may have to do with the concepts of floorEnd and ceilingEnd that are described in MazamaCoreUtils::dateRange.

[tabrasel]

It looks like the problem lies in the creation of the plot's X axis scale by custom_datetimeScale(), specifically this portion which extends the horizontal axis:

# Calculate padding ----------------------------------------------------------

# NOTE:  X-axis must be extended to fit the complete last day.
# NOTE:  Then a little bit more for style.
xRangeSecs <- as.numeric(difftime(enddate, startdate, timezone, units = "secs"))
marginSecs <- 0.02 * xRangeSecs
xlo <- startdate - lubridate::dseconds(marginSecs)

if (includeFullEnddate) { # TRUE by default
  xhi <-
    lubridate::floor_date(enddate, unit = "day") +
    lubridate::dhours(23) +
    lubridate::dseconds(marginSecs)
} else {
  xhi <- enddate + lubridate::dseconds(marginSecs)
}

If the given date range is 2016-07-1 to 2016-07-10, then xhi will be set to a bit before midnight on July 10th. However, monitor_ggDailyBarplot_archival() edits the user's given enddate before this with ceilingEnd=TRUE:

dateRange <- MazamaCoreUtils::dateRange(
  startdate = startdate,
  enddate = enddate,
  timezone = timezone,
  unit = "day",
  ceilingEnd = TRUE
)

startdate <- dateRange[1]
enddate <- min(c(dateRange[2], lubridate::now(tzone = timezone)))

With ceilingEnd=TRUE, enddate is changed from 2016-07-10 to 2016-07-11. When custom_datetimeScale() is called after this, it will create extra space for the new enddate which is one more than we need, and an empty gray bar will be drawn in the extra space.

POSSIBLE FIXES
We can't call daterange() with ceilingEnd=FALSE since that would cut off the data for the last day's mean aggregation. We also can't directly supply custom_datetimeScale() with the user's original enddate because it is imbedded in a rather lengthy callstack whose other functions require the edited enddate:

monitor_ggDailyBarplot_archival()
L ggplot_pm25Timeseries()
   L custom_pm25TimeseriesScales()
      L custom_datetimeScale()

One way I've found to avoid drawing the extra day in the x scale is to call custom_datetimeScale() with includeFullEnddate=FALSE. This can be done by setting that param in monitor_ggDailyBarplot_archival()'s call to ggplot_pm25Timeseries() (which filters down the param through the callstack). I don't really like this solution though since technically the result DOES include the full end date. Another possible solution would be to edit the xhi calculation to be 11:59:59 PM the night before enddate instead of near the end of enddate, effectively giving us the user's original enddate value:

xhi <-
    lubridate::floor_date(enddate, unit = "day") -
    lubridate::dseconds(1)

Unfortunately, while this fixes the problem for the barplot, it changes the core behavior of the datetimeScale function (and makes no logical sense).

I'll continue to look for better ways to fix this issue. Ideally I don't want to touch anything outside of monitor_ggDailyBarplot_archival(), but given the deeply nested function calls within it, that might not be possible.