Revert all commits after v0.5.417 (restore v0.5.417 tree state)

[maluchari]

Reverts the 17 commits that landed on main after v0.5.417, bringing the tree back to exactly the v0.5.417 state. Squashed into a single revert commit for reviewability and to allow reinstating everything as one unit (revert this commit to bring all 17 changes back).

Reverted commits (v0.5.417..main, newest first):

• Revert Support returning HTS fields in table list api #579 (HTS fields in table list api) (Revert #579 (HTS fields in table list api) #610)
• feat(optimizer): [3/N] Analyzer (feat(optimizer): [3/N] Analyzer #533)
• [DataLoader] Handle Cast(Literal, TIMESTAMP/DATE/TIME) in scan optimizer ([DataLoader] Support more literal types in filters DSL to DataFusion conversion #569 follow-up) ([DataLoader] Handle Cast(Literal, TIMESTAMP/DATE/TIME) in scan optimizer (#569 follow-up) #583)
• Skip metadata.json parse in drop path (Skip metadata.json parse in drop path #589)
• feat(optimizer): [2/N] Optimizer REST Service and Controller (feat(optimizer): [2/N] Optimizer REST Service and Controller #531)
• [BDP-102028] feat(optimizer): [1/N] Optimizer Database ([BDP-102028] feat(optimizer): [1/N] Optimizer Database #530)
• [RTAS]: Fix bug - remove fs scheme from tableLocation in commit (cont) ([RTAS]: Fix bug - remove fs scheme from tableLocation in commit (cont) #594)
• Trigger ELR process (Trigger ELR process #593)
• [BDP-102028] feat(optimizer): [0/N] Optimizer API and internal model ([BDP-102028] feat(optimizer): [0/N] Optimizer API and internal model #527)
• Fail retention app when the columnPattern mismatch partition spec (Fail retention app when the columnPattern mismatch partition spec for… #552)
• [DataLoader] Drop OpenTelemetry minimum version to 1.38.0 ([DataLoader] Drop OpenTelemetry minimum version to 1.38.0 #590)
• [DataLoader] Emit OpenTelemetry metrics for read operations ([DataLoader] Emit OpenTelemetry metrics for read operations #582)
• Cache iceberg metadata to reduce redundant requests to storage (Cache iceberg metadata to reduce redundant requests to storage #509)
• bump iceberg 1.2 version to 1.2.0.17 (bump iceberg 1.2 version to 1.2.0.17 #587)
• Support returning HTS fields in table list api (Support returning HTS fields in table list api #579)
• [DataLoader] Add unique id property to OpenHouseDataLoader ([DataLoader] Add unique id property to OpenHouseDataLoader #580)
• [DataLoader] Add OpenTelemetry metrics support ([DataLoader] Add OpenTelemetry metrics support #575)

Summary

Issue] Briefly discuss the summary of the changes made in this
pull request in 2-3 lines.

Changes

• Client-facing API Changes
• Internal API Changes
• Bug Fixes
• New Features
• Performance Improvements
• Code Style
• Refactoring
• Documentation
• Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

• Manually Tested on local docker setup. Please include commands ran, and their output.
• Added new tests for the changes made.
• Updated existing tests to reflect the changes made.
• No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
• Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Additional Information

• Breaking Changes
• Deprecations
• Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.