Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12871

Implement front end using Calcite (epic for v1)

    XMLWordPrintableJSON

Details

    • Epic
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • None
    • None
    • Frontend
    • None
    • Implement front end using Calcite
    • ghx-label-5

    Description

      Calcite provides a framework for compilation that will allow us to provide optimizations more easily and also allow more support for SQL standard queries.  This Epic will track the multiple commits needed to support this

      Attachments

        Issue Links

          is related to

          Task - A task that needs to be done. IMPALA-13563 Clean up logging with parameterized calls

          • Major - Major loss of function.
          • Resolved
          1.
          		 Support basic queries using Calcite on the frontend Sub-task Resolved Steve Carlin
          2.
          		 Import parser files from Calcite into Impala Sub-task Resolved Steve Carlin
          3.
          		 Allow function parsing for Impala Calcite planner Sub-task Resolved Steve Carlin
          4.
          		 Implement filtering conditions Sub-task Resolved Steve Carlin
          5.
          		 Create query option to choose the planner type Sub-task Resolved Steve Carlin
          6.
          		 Implement Values and Union Calcite RelNodes Sub-task Resolved Steve Carlin
          7.
          		 Implement Sort capability Sub-task Resolved Steve Carlin
          8.
          		 Calcite planner: Invert Joins should be implemented as a rule Sub-task Resolved Steve Carlin
          9.
          		 Calcite planner: Need to code Parquet Count Star Optimization Sub-task Resolved Steve Carlin
          10.
          		 Implement aggregation capability Sub-task Resolved Steve Carlin
          11.
          		 Need to make a more generic parser for evaluating stmts for the Calcite Planner Sub-task Resolved Steve Carlin
          12.
          		 Support Ranger Authorization for Calcite Planner. Sub-task Closed Fang-Yu Rao
          13.
          		 Need infrastructure to handle implicit casting for functions Sub-task Resolved Steve Carlin
          14.
          		 Better exception for Analyze*Expr class in Calcite Planner Sub-task Resolved Steve Carlin
          15.
          		 computeEvalCost needs better implementation in Calcite planner Sub-task Open Unassigned
          16.
          		 (Calcite) Implement tryConvertToTopN as a rule Sub-task Resolved Steve Carlin
          17.
          		 Implement analytic functions Sub-task Resolved Unassigned
          18.
          		 Calcite: fix and enable tpcds and tpch tests Sub-task Closed Unassigned
          19.
          		 Using Calcite planner for Iceberg tables crashes impalad Sub-task Resolved Steve Carlin
          20.
          		 Calcite planner: Outer join query with nonequi conjunct crashes Impala server Sub-task Resolved Steve Carlin
          21.
          		 Calcite planner: test_exprs test_expr_child_limit runs slow Sub-task Resolved Steve Carlin
          22.
          		 Make IN clause more efficient in Calcite Planner Sub-task Open Steve Carlin
          23.
          		 Implement ParserTest unit tests for Calcite Planner Sub-task Open Joe McDonnell
          24.
          		 Support explain statement in Calcite Planner Sub-task Resolved Steve Carlin
          25.
          		 Calcite planner does not take advantage of the hash partition of previous aggregates Sub-task Open Steve Carlin
          26.
          		 Calcite planner: convert expressions to normal form for performance Sub-task Resolved Steve Carlin
          27.
          		 Calcite planner: Fix issues with quotes Sub-task Resolved Steve Carlin
          28.
          		 Calcite planner: fix datetime/interval issues for tpcds queries Sub-task Resolved Steve Carlin
          29.
          		 Calcite planner: duplicate table in a query fails Sub-task Resolved Steve Carlin
          30.
          		 Calcite planner: Need some translation rules to get tpcds queries to work Sub-task Resolved Steve Carlin
          31.
          		 Calcite planner: fix aggregation.test queries Sub-task Resolved Steve Carlin
          32.
          		 Calcite Planner: IMPALA-1519 query throws exception Sub-task Resolved Steve Carlin
          33.
          		 Calcite planner: Add support for various analytic and agg functions Sub-task Resolved Steve Carlin
          34.
          		 Calcite planner: Bug fixes for an analytics.test Sub-task Resolved Unassigned
          35.
          		 Calcite Planner: some scalar subquery throws exception when handle single_value Sub-task Open Unassigned
          36.
          		 Calcite planner: group_concat failing with distinct Sub-task Resolved Steve Carlin
          37.
          		 Calcite planner: Make exceptions easier to classify Sub-task Resolved Joe McDonnell
          38.
          		 Crash in RawValue::PrintValue() when running query_test/test_chars.py Sub-task Reopened Steve Carlin
          39.
          		 Calcite planner: support sub-millisecond datetime parts Sub-task Resolved Steve Carlin
          40.
          		 Calcite planner: support decode function Sub-task Resolved Steve Carlin
          41.
          		 Calcite Planner: fix || operator (both concat and or) Sub-task Resolved Steve Carlin
          42.
          		 Calcite planner: support in clause coercing Sub-task Resolved Steve Carlin
          43.
          		 Calcite Planner: Handle functions taking literal types from a lower level Sub-task Resolved Steve Carlin
          44.
          		 Calcite planner: "real" type should be treated as double Sub-task Resolved Steve Carlin
          45.
          		 Calcite planner: Derive default decimal precision for functions Sub-task Resolved Steve Carlin
          46.
          		 Calcite planner: support for functions in exprs.test Sub-task Resolved Steve Carlin
          47.
          		 Calcite planner: handle escaped characters in string literal Sub-task Resolved Steve Carlin
          48.
          		 Calcite planner: run unsupported query options through original planner Sub-task Resolved Steve Carlin
          49.
          		 Calcite planner: wrong results when intersect mixed with union Sub-task Resolved Steve Carlin
          50.
          		 Calcite planner: Fix explicit_cast regression in tpcds 32 Sub-task Resolved Steve Carlin
          51.
          		 Calcite planner is falling back incorrectly for some select statements Sub-task Resolved Steve Carlin
          52.
          		 Calcite Planner: Handle Join Parsing issues Sub-task Resolved Steve Carlin
          53.
          		 Calcite Planner: Handle straight_join hint Sub-task Open Unassigned
          54.
          		 Calcite Planner: Fix exception when null is in values clause Sub-task Resolved Steve Carlin
          55.
          		 Calcite Planner: Handle Filter above Sort Sub-task Resolved Steve Carlin
          56.
          		 Calcite Planner: Push filter rel nodes through project rel nodes Sub-task Resolved Steve Carlin
          57.
          		 Calcite Planner: remove empty sort keys Sub-task Resolved Steve Carlin
          58.
          		 Calcite planner: Allow limit clause in subquery Sub-task Resolved Steve Carlin
          59.
          		 Calcite Planner: Exception being thrown on literal value check Sub-task Resolved Unassigned
          60.
          		 Calcite planner: Need special code for null casting to a decimal Sub-task Resolved Steve Carlin
          61.
          		 Calcite Planner: return proper labels for columns Sub-task Resolved Steve Carlin
          62.
          		 Calcite planner: outer join not aggregating nulls properly Sub-task Resolved Steve Carlin
          63.
          		 Create hooks in Frontend to call Calcite planner Sub-task Resolved Steve Carlin
          64.
          		 Calcite Planner: Implement hooks from Impala planner Sub-task Resolved Steve Carlin
          65.
          		 Calcite planner: Ndv with second argument causing server to crash Sub-task Open Unassigned
          66.
          		 Calcite planner: add "different exception msg" to test report Sub-task Resolved Steve Carlin
          67.
          		 Calcite planner: Temporarily disable handling tables with complex types Sub-task Resolved Steve Carlin
          68.
          		 Calcite Planner: Disable Calcite's constant folding Sub-task Reopened Steve Carlin
          69.
          		 Calcite planner: Enable Impala constant folding Sub-task Open Steve Carlin
          70.
          		 Calcite Planner: Fix test_date_queries test Sub-task Open Steve Carlin
          71.
          		 Calcite Planner: Queries with TupleIsNullPredicate failing for analytic functions Sub-task Resolved Steve Carlin
          72.
          		 Calcite Planner: enable UTF8 character set Sub-task Resolved Steve Carlin
          73.
          		 Calcite Planner: Implement partition key scan optimization Sub-task Resolved Steve Carlin
          74.
          		 Calcite Planner: implement handling of acid data Sub-task Resolved Steve Carlin
          75.
          		 Calcite Planner: Improper casting on char types Sub-task Resolved Steve Carlin
          76.
          		 Calcite planner, need to support DIV Sub-task Open Unassigned
          77.
          		 returnsAtMostOneRow should be part of the planner interface Sub-task Open Unassigned
          78.
          		 Enable hooks in test framework to run parser tests for Calcite planner Sub-task Open Joe McDonnell
          79.
          		 Calcite Planner: Run parser junit tests through Calcite Planner Sub-task Open Joe McDonnell
          80.
          		 Calcite Planner: Support views for the Calcite planner in Impala Sub-task Resolved Fang-Yu Rao
          81.
          		 Calcite Planner: Support Union for partition key scan Sub-task Open Unassigned
          82.
          		 Enable some plan tests for Calcite Planner Sub-task Resolved Unassigned
          83.
          		 Calcite Planner: add basic rules for performance Sub-task Open Steve Carlin
          84.
          		 Calcite Planner: Support queries against tables in the local catalog mode Sub-task Resolved Riza Suminto
          85.
          		 Calcite planner: Use table and column statistics for optimization Sub-task Open Steve Carlin
          86.
          		 Calcite Planner: Use pruned row counts for stats Sub-task Open Unassigned
          87.
          		 Calcite Planner: Add Calcite cost model calculations Sub-task Open Steve Carlin
          88.
          		 Calcite planner: Customize LoptOptimizeJoinRule Sub-task Open Steve Carlin
          89.
          		 Calcite planner: Runtime filters not being applied with outer joins Sub-task Open Steve Carlin
          90.
          		 Calcite planner: Register equivalent union expressions in value transfer graph Sub-task Open Unassigned
          91.
          		 Calcite Planner: Improper creation of multijoin Sub-task Resolved Steve Carlin
          92.
          		 Calcite planner: Add cbo validation to junit tests Sub-task Open Steve Carlin
          93.
          		 Calcite planner: Add top-n optimization for rank analytic expressions Sub-task Open Unassigned
          94.
          		 Calcite planner: parser error when run sqls in DBeaver Sub-task Open Unassigned
          95.
          		 Calcite Planner: varchar(6) being recast as string incorrectly Sub-task Open Steve Carlin
          96.
          		 Support IGNORE NULLS for functions Sub-task Open Fang-Yu Rao
          97.
          		 Calcite Planner: support SQL-standardized EXTRACT-FROM syntax Sub-task Open Unassigned
          98.
          		 Calcite Planner: support SQL-standardized TRIM-FROM syntax Sub-task Open Unassigned
          99.
          		 Calcite planner: nulls last not being handled correctly for analytical function Sub-task Open Unassigned
          100.
          		 Calcite planner: Type coercion code accidentally omitted from Analysis Sub-task Open Steve Carlin
          101.
          		 Calcite Planner: Ensure 'unsupported' functions are handled correctly Sub-task Open Unassigned
          102.
          		 Calcite Planner: union return different result Sub-task Open Unassigned
          103.
          		 Calcite Planner: Could not find the Impala function for COUNT/MIN Sub-task Closed Unassigned
          104.
          		 SqlValidatorException: Invalid number of arguments to function 'ILIKE'. Was expecting 0 arguments Sub-task Open Unassigned
          105.
          		 Calcite Planner: Impala needs to implement a converter that transforms COUNT() FILTER into a form that Impala can handle. Sub-task Closed Unassigned
          106.
          		 Calcite planner: Some erroneous cast functions are succeeding Sub-task Open Steve Carlin
          107.
          		 Calcite Planner: throw NullPointerException in ImpalaAggRel.createMappedOutputExprs Sub-task Open Steve Carlin
          108.
          		 Calcite planner: Investigate filter selectivity for join non nulls Sub-task Open Unassigned
          109.
          		 Calcite Planner: select max(1); failed: IllegalStateException: null Sub-task Open Unassigned
          110.
          		 Calcite planner: Wrong cardinality shown with is_not_null predicate in scan node Sub-task Open Unassigned
          111.
          		 Calcite Planner: Produce column lineage graph when Calcite is the planner Sub-task Open Fang-Yu Rao
          112.
          		 Calcite Planner: Support queries against virtual columns Sub-task Open Fang-Yu Rao
          113.
          		 Calcite planner: Need support for LocalFsTable tables Sub-task Resolved Steve Carlin
          114.
          		 Calcite Planner: reduce the calls to Class.forName, as this can create a performance bottleneck under high concurrency Sub-task Open Unassigned
          115.
          		 Calcite planner: labels for expressions not matching original planner Sub-task Open Steve Carlin
          116.
          		 Calcite Planner: server flag should run Calcite through JniFrontend Sub-task Open Unassigned
          117.
          		 Calcite planner: support statement_expression_limit query option Sub-task Open Unassigned
          118.
          		 Common table expression (CTE) in a WITH clause should not be treated as a table by TableVisitor for the Calcite planner Sub-task Resolved Fang-Yu Rao
          119.
          		 Calcite Planner: Calcite treating exponent notation as double Sub-task Open Unassigned
          120.
          		 Calcite planner: enable Disable_outermost_topn query option Sub-task Open Unassigned
          121.
          		 Calcite Planner: v1.40 of IntersectToDistinctRule throws exception Sub-task Open Unassigned
          122.
          		 Calcite planner: case statement returning wrong types for char, varchar Sub-task Open Unassigned
          123.
          		 Calcite planner: Crash on Union with underlying Project and Values nodes Sub-task Open Unassigned
          124.
          		 Support column masking and row filtering for Calcite planner Sub-task Open Fang-Yu Rao
          125.
          		 Calcite planner: Use the constant folding within Impala instead of Calcite Sub-task Resolved Unassigned
          126.
          		 Calcite Planner: 1.40 breaks coercion of smaller numeric datatypes Sub-task Open Unassigned
          127.
          		 Calcite Planner: Fix In operator in v1.40 Sub-task Open Unassigned
          128.
          		 Upgrade Calcite to v1.40 Sub-task Open Unassigned
          129.
          		 Calcite Planner: Implement Optimized Partition Scan feature Sub-task Open Unassigned
          130.
          		 Calcite planner: Emulate Impala one row right side join logic Sub-task Open Unassigned
          131.
          		 Calcite Planner: Implement single row join optimization Sub-task Open Unassigned
          132.
          		 Calcite planner: Missed parquet statistics predicates Sub-task Open Unassigned
          133.
          		 Calcite Planner: Expand the processing scope of the ExtractLiteralAgg rule Sub-task Open Unassigned
          134.
          		 Calcite planner: Need a better estimate for deciding if a runtime filter is needed Sub-task Open Unassigned
          135.
          		 Remove avatica dependency from jar file Sub-task Open Unassigned
          136.
          		 Calcite planner: allow semi-colon in direct connection Sub-task Open Steve Carlin
          137.
          		 Calcite planner: ordering by "random()" fails Sub-task Open Steve Carlin
          138.
          		 Calcite planner: Add column stats to union slot descriptors Sub-task Open Steve Carlin
          139.
          		 Calcite planner: Change test files to match Calcite results Sub-task Open Unassigned
          140.
          		 Calcite planner: double partition types is failing after constant folding Sub-task Open Steve Carlin
          141.
          		 Calcite planner: handle escaped double quote character Sub-task Open Steve Carlin
          142.
          		 Calcite planner: support fallback to Original planner Sub-task Open Steve Carlin
          143.
          		 Calcite Planner: Add support for RexSimplify Sub-task Open Steve Carlin
          144.
          		 Calcite Planner: Remove CalciteJniFrontend class Sub-task Open Unassigned
          145.
          		 Calcite planner allowing cast of date to int Sub-task Open Steve Carlin
          146.
          		 Calcite planner returning wrong estimates for is null predicates Sub-task Open Unassigned
          147.
          		 Calcite planner: Limit needs to be applied to select node Sub-task Open Unassigned

          Activity

            People

              scarlin Steve Carlin
              scarlin Steve Carlin
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: