Spark Connect
It is guaranteed that there is at least one ARROW batch returned even if the result set is empty¶
POST/api/v2/spark-connect/execute-plan
Executes a request that contains the query and returns a stream of [[ExecutePlanResponse]].
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |
Analyzes a query and return a [[AnalyzeResponse]] containing metadata about the query¶
POST/api/v2/spark-connect/analyze-plan
Analyzes a query and return a [[AnalyzeResponse]] containing metadata about the query.
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |
Update or fetch the configurations and returns a [[ConfigResponse]] containing the result¶
POST/api/v2/spark-connect/config
Update or fetch the configurations and returns a [[ConfigResponse]] containing the result.
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 | Successfully accepted the request, but it is not completed yet. |
Add artifacts to the session and returns a [[AddArtifactsResponse]] containing metadata about the added artifacts¶
POST/api/v2/spark-connect/add-artifacts
Add artifacts to the session and returns a [[AddArtifactsResponse]] containing metadata about the added artifacts.
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |
Check statuses of artifacts in the session and returns them in a [[ArtifactStatusesResponse]]¶
POST/api/v2/spark-connect/artifact-status
Check statuses of artifacts in the session and returns them in a [[ArtifactStatusesResponse]]
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |
Interrupts running executions¶
POST/api/v2/spark-connect/interrupt
Interrupts running executions
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |
The ExecutePlan must have been started with ReattachOptions.reattachable=true. If the ExecutePlanResponse stream ends without a ResultComplete message, there is more to continue. If there is a ResultComplete, the client should use ReleaseExecute with¶
POST/api/v2/spark-connect/reattach-execute
Reattach to an existing re-attachable execution.
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |
The ExecutePlan must have been started with ReattachOptions.reattachable=true. Non re-attachable executions are released automatically and immediately after the ExecutePlan RPC and ReleaseExecute may not be used¶
POST/api/v2/spark-connect/release-execute
Release an re-attachable execution, or parts thereof.
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |
It is used for sproc dataframe processor to pull Spark request from GS¶
POST/api/v2/spark-connect/pull-request
Get the next Spark request.
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |
It is used for sproc dataframe processor to push Spark response to GS¶
POST/api/v2/spark-connect/push-response
Executes a request from sproc that contains the Spark response.
Response¶
| Code | Description |
|---|---|
| 200 | successful |
| 202 |