Verify & fix tutorials against Hopsworks 5.0#350
Open
MagicLex wants to merge 3 commits into
Open
Conversation
5.0 prepended a `logging_data` parameter to FeatureView.log(), shifting positional args. The inference notebook passed untransformed features and predictions positionally, so they landed in the wrong slots and logging raised FeatureStoreException. Pass them by keyword. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s/example 5.0 enforces that a feature view join covers every column of the joined feature group's primary key (errorCode 270249). The previous_applications, pos_cash_balances, installment_payments and credit_card_balances groups are at (sk_id_prev, sk_id_curr) grain but were joined on sk_id_curr only, so FV creation was rejected. - functions.py: add collapse_to_customer() — reduce a previous-credit table to one row per sk_id_curr (most recent by datetime), dropping sk_id_prev. - nb1: apply it before the four inserts; set their primary key to [sk_id_curr]. - nb3: drop sk_id_prev from the four select_except lists so the join is on the full primary key. Two pre-existing data-hygiene issues surfaced once the FV builds: - the bureaus-rooted left join leaves a null target for bureau records whose applicant is absent from applications; drop those rows before training. - the input_example carried pd.NA (encoded null categoricals); sample a complete row. Note: running this end-to-end also requires the hsfs builtin label_encoder null-safety fix (separate SDK change). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This Prophet tutorial was not runnable end to end. Fixes: - 5.0 renames transformed columns to `<transform>_<feature>_` and prefixes joined event-time columns with `<project>_<fg>_<version>_`. Add restore_column_names() after train_test_split / get_batch_data so the `date` column and the per-feature Prophet regressors resolve again. - Prophet rejects NaN in regressor columns; drop rows with a missing regressor or label before fit/predict (train, test and batch). - nb2: the MAE cell referenced `forecast` before the predict cell defined it; reorder predict before MAE. - nb2: matplotlib was imported as `from matplotlib import pyplot` but the plot helpers use `plt`; import as `plt`. - input_example: sample a complete row (encoded nulls are pd.NA, not JSON serializable). Running end-to-end also requires the hsfs builtin label_encoder null-safety fix (separate SDK change). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Running the tutorials end-to-end against 5.0.0.dev1 and fixing what breaks. One commit per logical fix.
Batch AI systems — verified end-to-end on a live 5.0 cluster
feature_view.log()called positionally; 5.0 prepended alogging_dataparam, so args landed in the wrong slots. Pass by keyword.previous_*groups are (sk_id_prev, sk_id_curr) grain but joined on sk_id_curr only. Collapse them to applicant grain; handle null labels/input_example.pltimport.Depends on an SDK fix
credit_scores and hospital_wait_time only run once the hsfs builtin
label_encoderis made null-safe (1-line). Tracked separately.Still to verify
real-time, llm, api_examples, integrations.
Note: branch-5.0 is a protected pattern, so this targets master.
🤖 Generated with Claude Code