DataFlux Data Management Batch Job REST API
Version
This is version 1 of this Representational State Transfer (REST) Application Programming Interface (API).
Overview
The DataFlux Data Management Batch Job REST API is used to interact with (manage and execute) data management batch jobs that reside on a DataFlux Data Management Server.
This API enables clients to perform the following tasks:
- list, upload, download, update, or delete batch job flows on the server
- retrieve (as inputs) the metadata of individual batch job flows
- execute individual batch jobs by passing input data to them
- retrieve the status of the execution of batch jobs (and the corresponding log files)
DataFlux Data Management Server supports requests through REST and SOAP APIs; both are handled by the same common layer. The DataFlux Data Management Batch Job REST API is an alternative to the older SOAP API of the DataFlux Data Management Server. This REST API is designed to expose the preexisting functionality in a REST-compliant way.
If DataFlux Data Management Server security is enabled, the only authentication mechanism that is supported by the server is HTTP Basic authentication. This means that every request will include base64-encoded credentials. Depending on your site's security requirements, you might choose to enable the
SSL
option on DataFlux Data Management Server to force all requests to come over an encrypted connection.
If DataFlux Data Management Server security is enabled, authorization checks are performed when the specific request is made (not before). This means that a client that requests a list of job flows will see all of the job flows, even those to which the client has no access. This is similar to Windows or UNIX, where you can see all of the files in a directory, even though permissions on some of the files prevent you from accessing them.
The current implementation supports only JSON (JavaScript Object Notation) as a data exchange format. The only exception to this is a job flow content media type, which is an opaque XML document.
Some REST HTTP clients do not send a POST request without a body. Therefore, it is advised to set the body to
{}
when executing a job flow without input data.
The current implementation does not support Conditional PUT for the GET, HEAD, and PUT operations.
Usage Scenario
In a typical usage scenario, the following actions occur:
- A user designs a batch job flow in a desktop DataFlux Data Management Studio client.
- The user pushes the batch job flow out to DataFlux Data Management Server. (A job flow exists on the DataFlux Data Management Server as an XML file.)
- The batch job flow is visible to all REST clients of the server, who can now execute it, potentially making the batch job flow a part of their business data processing logic. (Executing a batch job flow means that REST clients send a request body that can contain inputs for the job, and they get back a response with the status of the batch job run.)
- DataFlux Data Management Server receives a request to execute a batch job flow.
- When the DataFlux Data Management Server receives a request to execute the job flow, it starts a child process (or reuses an idle child process) that loads the DataFlux Data Management Platform job flow processing engine (which runs DataFlux Data Management jobs).
- The job flow processing engine loads the specified job flow, sets any received inputs for the loaded job flow, and executes it. (It passes any received inputs to the child process.) As data flows through the nodes, the data is transformed or augmented, as the nodes of the job flow dictate. The job flow execution might or might not result in the generation of output data.
- DataFlux Data Management Server associates a unique jobId with the executing batch job. The jobId is returned to the waiting client. The response contains a job run status representation that includes this jobID element. (This ID can be of variable length, and contains only characters that are valid for a URL.) Whether the job is still running or has already finished, the jobId can be used to request the status of that specific job run instance or its log. It can also be used to cancel the job execution.
- Clients interrogate DataFlux Data Management Server for the statuses of running and finished batch jobs. The clients can do the following:
- continue interrogating the server for updates to the status of that batch job run, optionally obtaining the job log file while the job is running or after it has finished.
- cancel a job execution (A jobId ceases to be valid only when a client explicitly deletes the status of that specific job execution instance from the server.)
- delete the history of the job execution (that is, the job log file).
Scope
This public API covers the batch jobs functionality of DataFlux Data Management Server. It coexists with other APIs of the product that are focused on the real-time services execution and administration services:
- DataFlux Data Management Real-Time Data Job REST API
- DataFlux Data Management Real-Time Process Job REST API
Base URL
Here is the base URL for batch jobs:
http://www.example.com/SASDataMgmtBatchJob/rest
Media Types
Externally-Defined Media Types
The following table lists externally-defined media types. These are standard SAS media types, the description of which can be obtained from your SAS consultant or SAS Technical Support. These media types are not specific to DataFlux batch jobs.
application/vnd.sas.datamgmt.jobflow.content+xml
The application/vnd.sas.datamgmt.jobflow.content+xml media type is the XML document that contains a job flow. A job flow is an XML document and cannot be anything else. This media type is really just an alias for application/xml. It has no link relations or members. It is an opaque XML string with no publicly documented schema. The job flow XML document is generated and manipulated by DataFlux Data Management Studio. It is not manipulated by external clients.
application/vnd.sas.datamgmt.jobflow
The application/vnd.sas.datamgmt.jobflow media type defines a job flow, which is composed of a job flow name string and the job flow XML content. It is used to create new job flows on the server. It has no link relations. It is just a container for name and content elements.
Member Name | Type | Description |
---|---|---|
version |
integer
|
This media type's schema version number. This representation is version 1. |
name
|
string
|
The name of this job flow (including relative path components, if any). A string of up to 4096 bytes in length, which can contain only characters that are valid in a filename under both Windows and UNIX. |
content
|
string
|
A base64-encoded job flow content XML string. (application/vnd.sas.datamgmt.jobflow.content+xml) |
JSON Representation Example
{ "version": 1,
"name" : "Event for Logging.djf",
"content" : "PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9Im5vIiA/yb3A+DQogICAgPC9wcm9wc2V0Pg=="
}
application/vnd.sas.datamgmt.jobflow.definition.summary
The application/vnd.sas.datamgmt.jobflow.definition.summary media type defines a summary of a job flow. It is returned as elements of a collection of all available job flows. It supports browse functionality (that is, it shows which job flows are available on the server).
Link Relation | HTTP Method | Description |
---|---|---|
self |
GET
|
Get the definition of a specific job flow, given its ID. URI: Media type: application/vnd.sas.datamgmt.jobflow.definition |
delete
|
DELETE
|
Delete a specific job flow, given its ID. URI: |
Member Name | Type | Description |
---|---|---|
version |
integer
|
This media type's schema version number. This representation is version 1. |
id
|
string
|
The system-assigned unique ID for this object. |
name |
string
|
The name of this job flow. It is provided by the client when issuing a POST request for the content of a new job flow. It cannot be changed. |
description
|
string
|
The description of this job flow. It is an element of the job flow content and cannot be changed. It is returned only if the value is available to the server at the time of the request. |
links
|
array of link objects |
Zero or more link objects. For a description of the link types, see the Link Relations table above. |
JSON Representation Example
{ "version": 1,
"id" : "RXZlbnQgZm9yIExvZ2dpbmcuZGpm",
"description" : "Sample service to log generated events",
"name" : "Event for Logging.djf",
"links": [
{
"href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.datamgmt.jobflow.definition+json",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm"
},
{
"href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm",
"method": "DELETE",
"rel": "delete",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm"
}
]
}
application/vnd.sas.datamgmt.jobflow.definition
The application/vnd.sas.datamgmt.jobflow.definition media type is the full representation of a job flow definition. This media type is used to get the metadata of a job flow, manipulate a job flow, and execute a job flow.
Link Relation | HTTP Method | Description |
---|---|---|
self |
GET
|
Get the definition of the job flow. URI: Media type: application/vnd.sas.datamgmt.jobflow.definition |
delete
|
DELETE
|
Delete the job flow. URI:
|
metadata
|
GET
|
Get the metadata of the job flow (input and outputs). URI:
Media type: application/vnd.sas.datamgmt.jobflow.metadata |
content
|
GET
|
Get the contents of the job flow (the XML document). URI: Media type: application/vnd.sas.datamgmt.jobflow.content+xml |
updateContent
|
PUT
|
Update the contents of the job flow (the XML document). URI: Media type: application/vnd.sas.datamgmt.jobflow.content+xml |
execute
|
POST
|
Execute the job flow. (Inputs are passed in; outputs are returned.) URI:
Media type: application/vnd.sas.datamgmt.jobflow.metadata |
jobs
|
GET
|
Get the collection of application/vnd.sas.datamgmt.job.status executions of this job flow. URI: Media type:application/vnd.sas.collection |
up
|
GET
|
Go back to the list of available job flows. URI:
Media type: application/vnd.sas.collection |
Member Name | Type | Description |
---|---|---|
version |
integer
|
This media type's schema version number. This representation is version 1. |
id
|
string
|
The system-assigned unique ID for this object. |
name |
string
|
The name of this job flow. It is provided by the client when issuing a POST request for a new job flow content. It cannot be changed. |
description
|
string
|
The description of this job flow. It is an element of the job flow content and cannot be changed. It is returned only if the value is available to the server at the time of the request. |
size |
integer
|
The size of this job flow (XML document) in bytes. |
modifiedTimeStamp
|
string
|
Last modification time of this job flow. |
links
|
array of link objects |
Zero or more link objects. For a description of the link types, see the Link Relations table above. |
JSON Representation Example
{ "version": 1,
"id" : "RXZlbnQgZm9yIExvZ2dpbmcuZGpm",
"name" : "Event for Logging.djf",
"description" : "",
"modifiedTimeStamp": "20150106T16:16:37",
"size": 3432,
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.datamgmt.jobflow.definition+json",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm",
"method": "DELETE",
"rel": "delete",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm/metadata",
"method": "GET",
"rel": "metadata",
"type": "application/vnd.sas.datamgmt.jobflow.metadata+json",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm/metadata"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm/content",
"method": "GET",
"rel": "content",
"type": "application/vnd.sas.datamgmt.jobflow.content+xml",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm/content"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm/content",
"method": "PUT",
"rel": "updateContent",
"type": "application/vnd.sas.datamgmt.jobflow.content+xml",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm/content"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm",
"method": "POST",
"rel": "execute",
"type": "application/vnd.sas.datamgmt.jobflow.metadata+json",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm/jobs",
"method": "GET",
"rel": "jobs",
"type": "application/vnd.sas.collection+json",
"uri": "/jobFlowDefns/RXZlbnQgZm9yIExvZ2dpbmcuZGpm/jobs"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns",
"method": "GET",
"rel": "up",
"type": "application/vnd.sas.collection+json",
"uri": "/jobFlowDefns"
}
]
}
application/vnd.sas.datamgmt.jobflow.metadata
The application/vnd.sas.datamgmt.jobflow.metadata media type describes the inputs that the given job flow accepts and the outputs that it can generate. It is also used as a container for the input data that is sent in the execute
request and as a container for the output data that is returned from that request. When the client gets a batch job flow metadata representation, it knows how to construct the inputs container for the execute
request and what to expect in the outputs container of the response.
Note: No outputs are returned from the execute
request.
Link Relation | HTTP Method | Description |
---|---|---|
execute |
POST
|
Execute the job flow. (Inputs are passed in; outputs are returned.) URI: Media type: application/vnd.sas.datamgmt.jobflow.metadata |
up
|
GET
|
Go back to the definition of the job flow.
URI: Media type: application/vnd.sas.datamgmt.jobflow.definition |
self
|
GET
|
Refresh this metadata. URI:
Media type: application/vnd.sas.datamgmt.jobflow.metadata |
Member Name | Type | Description |
---|---|---|
version
|
integer
|
This media type's schema version number. This representation is version 1. |
id
|
string
|
The system-assigned unique ID for this object. |
name |
string
|
The name of this job flow. It is provided by the client when issuing a POST request for a new job flow content. It cannot be changed. |
description
|
string |
The description of this job flow. It is an element of the job flow content and it cannot be changed. |
inputs
|
object
|
This is a set of bindings: an unordered set of zero or more name-value pairs and zero or more data tables. It can be used to construct the inputs member of the request to execute a job flow. The name-value pairs are always strings, where the value can be an empty string or null. It is valid to have no inputs member in the request (in which case the job flow is executed with no input data). The types are defined by DataFlux Data Management Platform (not DataFlux Data Management Server). The data values are converted to strings before they are passed to DataFlux Data Management Platform for job flow execution. The following types are supported:
A data table is a dynamic type that contains two objects. One is metadata, defining an array of names of columns and the type of data stored in each. The other is data, containing an array of rows with columns of data identical to those defined in the metadata.
A data table always has at least one column, and can have zero or more rows. The data values in rows can be null or contain empty strings (for columns of string type). |
outputs
|
object
|
This is a set of bindings: an unordered set of zero or more name-value pairs and zero or more data tables. It can be used to interpret the outputs member of the response from a service data flow execution request. It is valid to have no outputs member in the response (which means the executed service job flow produced no output data). For more information, see the inputs member above. |
links |
array of link objects
|
Zero or more link objects. For a description of the link types, see the Link Relations table above. |
JSON Representation Example
{ "version": 1
"id": "ZGF0YVN2Y1R5cGVzLmRkZg==",
"description": "a data service that exposes all available data types",
"name": "dataSvcTypes.ddf",
"inputs": {
"Process Priority": "low",
"Logging": "disable",
"dataTable": {
"metadata": [
{"name": "Full Name", "type": "string", "maxChars": 255},
{"name": "Age", "type": "integer"},
{"name": "Donor", "type": "boolean"},
{"name": "Birthday", "type": "date"},
{"name": "Temperature", "type": "decimal"}
],
"data": [
["sample string", 2, true, "2014-10-13", 5.5],
["another string", 246, false, "2014-12-31", 15.7075]
]
}
},
"outputs": {
"Address": null,
"RecordsProcessed": "2",
"dataTable": {
"metadata": [
{"name": "status", "type": "string", "maxChars": 15},
{"name": "status code", "type": "integer"}
],
"data": [
["success", 1],
["", null]
]
}
},
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/ZGF0YVN2Y1R5cGVzLmRkZg==",
"method": "POST",
"rel": "execute",
"type": "application/vnd.sas.datamgmt.jobflow.metadata+json",
"uri": "jobFlowDefns/ZGF0YVN2Y1R5cGVzLmRkZg=="
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/ZGF0YVN2Y1R5cGVzLmRkZg==",
"method": "GET",
"rel": "up",
"type": "application/vnd.sas.datamgmt.jobflow.definition+json",
"uri": "jobFlowDefns/ZGF0YVN2Y1R5cGVzLmRkZg=="
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/ZGF0YVN2Y1R5cGVzLmRkZg==/metadata",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.datamgmt.jobflow.metadata+json",
"uri": "jobFlowDefns/ZGF0YVN2Y1R5cGVzLmRkZg==/metadata"
}
]
}
application/vnd.sas.datamgmt.job.status
The application/vnd.sas.datamgmt.job.status media type defines the status of a job execution instance. It is returned as elements of a collection of all available job execution instances, or all job executions of a given job flow, or all job executions started by a given user.
For more information about the collections and filters, see the Resources section.
Link Relation | HTTP Method | Description |
---|---|---|
self |
GET |
Get the collection with this one job execution status, given its ID. URI: Media type:application/vnd.sas.collection |
jobFlow
|
GET |
Get the definition of a specific job flow, given its ID. URI: |
log |
GET
|
Get the job execution log file. URI: Media type: text/plain |
cancel |
PUT
|
Cancel the job execution. (That is, signal the engine that is running the job to cancel the run; no contents need to be included.) This is available only when the job is still running. URI: |
kill |
PUT
|
Kill the job. (That is, kill the engine running the job; no contents need to be included.) This is available only when the job is still running. URI: |
delete |
DELETE |
Delete the status of this job execution instance. This is available only when the job is no longer running. URI:
|
Member Name | Type | Description |
---|---|---|
version |
integer
|
This media type's schema version number. This representation is version 1. |
id |
string |
The system-assigned unique ID for this object. |
jobFlowId |
string |
The system-assigned unique job flow ID. |
name |
string
|
The name of this job flow. |
state |
string |
The state of this job execution (queued, running, completed, failed, or canceled). |
userName |
string |
The name of the user who executed this job. |
startTimeStamp |
string
|
The time stamp of the beginning of this job execution. |
stopTimeStamp |
string
|
The time stamp of the end of this job execution. If the job is still running, the value is an empty string. |
duration |
double
|
The number of seconds this job has been running (or how long it ran if it has finished running). |
links
|
array of link objects
|
Zero or more link objects. For a description of the link types, see the Link Relations table above. |
JSON Representation Example
{ "version": 1,
"jobFlowId": "MTAwRXhwK0NvcHkuZGRm",
"id": "MTQxNzU1MjY4NjoxNDpQVUJMSUM=",
"name": "100Exp+Copy.ddf",
"userName": "PUBLIC",
"state": "completed",
"startTimeStamp": "2014-12-02T16:04:32",
"stopTimeStamp": "2014-12-02T16:05:34",
"duration": 62,
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/MTAwRXhwK0NvcHkuZGRm",
"method": "GET",
"rel": "jobFlow",
"type": "application/vnd.sas.datamgmt.jobflow.definition+json",
"uri": "/jobFlowDefns/MTAwRXhwK0NvcHkuZGRm"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs/MTQxNzU1MjY4NjoxNDpQVUJMSUM=",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.collection+json",
"uri": "/jobs/MTQxNzU1MjY4NjoxNDpQVUJMSUM="
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs/MTQxNzU1MjY4NjoxNDpQVUJMSUM=/log",
"method": "GET",
"rel": "log",
"type": "text/plain",
"uri": "/jobs/MTQxNzU1MjY4NjoxNDpQVUJMSUM=/log"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs/MTQxNzU1MjY4NjoxNDpQVUJMSUM=",
"method": "DELETE",
"rel": "delete",
"uri": "/jobs/MTQxNzU1MjY4NjoxNDpQVUJMSUM="
}
]
}
Resources
Summary of Resources
Resource | Methods |
---|---|
/ |
|
/jobFlowDefns
|
|
/jobFlowDefns/{id}
|
|
/jobs
|
|
/serverInfo
|
GET /serverInfo
|
/engines
|
DELETE /engines
|
Resource /
The / resource returns a collection of links at the root of this REST API.
GET /
Method | Description |
---|---|
|
Return a collection of top-level links. Authentication required: True |
Request:
GET http://www.example.com/SASDataMgmtBatchJob/rest/
Response: A collection of links, using media type application/json.
{ "links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns",
"method": "GET",
"rel": "jobFlowDefns",
"type": "application/vnd.sas.collection+json",
"uri": "/jobFlowDefns"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns",
"method": "POST",
"rel": "create",
"type": "application/vnd.sas.datamgmt.jobflow+json",
"uri": "/jobFlowDefns"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs",
"method": "GET",
"rel": "jobs",
"type": "application/vnd.sas.collection+json",
"uri": "/jobs"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs?name={URL-encoded-name}",
"method": "GET",
"rel": "jobsByName",
"type": "application/vnd.sas.collection+json",
"uri": "/jobs?name={URL-encoded-name}"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs?user={URL-encoded-name}",
"method": "GET",
"rel": "jobsByUser",
"type": "application/vnd.sas.collection+json",
"uri": "/jobs?user={URL-encoded-name}"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs?state={jobState}",
"method": "GET",
"rel": "jobsByState",
"type": "application/vnd.sas.collection+json",
"uri": "/jobs?state={jobState}"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs?state={jobState}",
"method": "DELETE",
"rel": "deleteJobsByState",
"uri": "/jobs?state={jobState}"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/serverInfo",
"method": "GET",
"rel": "serverInfo",
"type": "application/json",
"uri": "/serverInfo"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/engines",
"method": "DELETE",
"rel": "delete",
"uri": "/engines"
}
],
"version": 1
}
Resource /jobFlowDefns
The /jobFlowDefns
resource is a collection of summaries of the job flows that are available on the server.
GET /jobFlowDefns
Method | Description |
---|---|
GET /jobFlowDefns |
List summaries of job flows that are available on the server. The collection typically contains a small number of items, perhaps up to 25. However, the number of items might be up to 100 in extreme cases. The summary objects themselves are small in terms of the amount of data contained in each one. Therefore, neither pagination nor filtering by job flow name components are supported in the initial version of the API. Authentication required: True |
Request:
GET http://www.example.com/SASDataMgmtBatchJob/rest/jobFlowDefns
Response: The response will contain a collection of application/vnd.sas.datamgmt.jobflow.definition.summary items.
{ "version": 1
"name": "items",
"count": 2,
"items": [
{ "description": "",
"id": "d2FpdGVyMi5kbWM=",
"name": "waiter2.dmc",
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/d2FpdGVyMi5kbWM=",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.datamgmt.jobflow.definition+json",
"uri": "/jobFlowDefns/d2FpdGVyMi5kbWM="
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/d2FpdGVyMi5kbWM=",
"method": "DELETE",
"rel": "delete",
"uri": "/jobFlowDefns/d2FpdGVyMi5kbWM="
}
]
},
{ "description": "",
"id": "d2FpdGVyLmRtYw==",
"name": "waiter.dmc",
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/d2FpdGVyLmRtYw==",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.datamgmt.jobflow.definition+json",
"uri": "/jobFlowDefns/d2FpdGVyLmRtYw=="
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/d2FpdGVyLmRtYw==",
"method": "DELETE",
"rel": "delete",
"uri": "/jobFlowDefns/d2FpdGVyLmRtYw=="
}
]
},
],
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/",
"method": "GET",
"rel": "up",
"type": "application/vnd.sas.link",
"uri": "/"
}
]
}
HTTP Response Codes: 200, 401, 403, 500
POST /jobFlowDefns
Method | Description |
---|---|
POST /jobFlowDefns |
Create a new job flow on the server with the given name and content using the application/vnd.sas.datamgmt.jobflow media type. Authentication required: True |
Note: DataFlux Data Management Server has no knowledge of the job flow content XML that is passed in and does not validate it. The Data Management Platform Engine runs the job flows that load and interpret that XML. The job flow XML is created by a UI tool such as DataFlux Data Management Studio, which ensures its validity.
- For the
POST
request, if an invalid job flow content XML is uploaded, the job flow fails to execute, with a specific error message from the Data Management Platform Engine detailing the cause of the problem. - For the
PUT
request as well, if invalid job flow content XML is uploaded when updating an existing job flow with new XML content, the job flow fails to execute, with a specific error message from the Data Management Platform Engine detailing the cause of the problem.
Note: DataFlux Data Management Server requires job flows to have unique names. If a job flow with the same name as the one being posted already exists, DataFlux Data Management Server returns HTTP error 409 .
Note: The job flow name component of the request allows a relative path as part of the job flow name. This causes DataFlux Data Management Server to create the specified job flow name under that relative path (creating the path itself along the way, if necessary).
Note: DataFlux Data Management Server controls what characters can be used in a valid job flow name. It is a lowest common denominator of characters valid in filenames on Windows and UNIX platforms. If an invalid character is found, DataFlux Data Management Server returns HTTP error 404. Because DataFlux Data Management Server controls the characters that are allowed in job flow names, basic URL encoding of the name is sufficient.
Request:
POST http://www.example.com/SASDataMgmtBatchJob/rest/jobFlowDefns
Response: The response will contain an application/vnd.sas.datamgmt.jobflow.definition item for the newly created job flow.
HTTP Response Codes: 200, 400, 401, 403, 404, 409 , 413, 500
Resource /jobFlowDefns/{id}
The /jobFlowDefns/{id}
resource is of media type application/vnd.sas.datamgmt.jobflow.definition. It allows for the manipulation and execution of the specified job flow.
HEAD /jobFlowDefns/{id}
Method | Description |
---|---|
HEAD /jobFlowDefns/{id} |
Check that the job flow exists. Authentication required: True |
Request:
HEAD http://www.example.com/SASDataMgmtBatchJob/rest/jobFlowDefns/{id}
Response: The response will contain no body.
HTTP Response Codes: 200, 400, 401, 404
GET /jobFlowDefns/{id}/content
Method | Description |
---|---|
GET /jobFlowDefns/{id}/content |
Get the contents of the job flow (that is, retrieve the XML document that is the job flow). Authentication required: True |
Request:
GET http://www.example.com/SASDataMgmtBatchJob/rest/jobFlowDefns/{id}/content
Response: The response will contain an XML document that is the job flow.
HTTP Response Codes: 200, 400, 401, 403, 404, 413, 500
PUT /jobFlowDefns/{id}/content
Method | Description |
---|---|
PUT /jobFlowDefns/{id}/content |
Update the contents of a job flow on the server with the job flow XML passed in the request. Note: This method changes the contents of a job flow, but not its name. The current version of this REST API does not support a job flow rename operation. To rename a job flow, upload it to DataFlux Data Management Server with a new, unique path name, and delete the old job flow. Authentication required: True |
Request:
PUT http://www.example.com/SASDataMgmtBatchJob/rest/jobFlowDefns/{id}/content
Response: The response will contain an application/vnd.sas.datamgmt.jobflow.definition item for the updated job flow.
HTTP Response Codes: 200, 400, 401, 403, 404, 413, 500
DELETE /jobFlowDefns/{id}
Method | Description |
---|---|
DELETE /jobFlowDefns/{id} |
Delete a job flow from the server. This does not affect jobs that are already running. (They continue running.) Job flows waiting in the queue to run log an HTTP error 404 after they are removed from the queue. Authentication required: True |
Request:
DELETE http://www.example.com/SASDataMgmtBatchJob/rest/jobFlowDefns/{id}
HTTP Response Codes: 204, 400, 401, 403, 404, 413, 500
GET /jobFlowDefns/{id}/metadata
Method | Description |
---|---|
GET /jobFlowDefns/{id}/metadata |
Get the metadata for the job flow. This is implemented as a dedicated request, rather than automatically being included in the application/vnd.sas.datamgmt.jobflow.definition media type. Obtaining metadata can be a very expensive operation (involving starting a child process, loading a job flow, and contacting databases). Therefore, the user must explicitly ask for this information when it is actually needed. Authentication required: True |
Request:
GET http://www.example.com/SASDataMgmtBatchJob/rest/jobFlowDefns/{id}/metadata
Response:
The response will contain an application/vnd.sas.datamgmt.jobflow.metadata item for the job flow.
HTTP Response Codes: 200, 400, 401, 403, 404, 500
GET /jobFlowDefns/{id}/jobs
Method | Description |
---|---|
GET /jobFlowDefns/{id}/jobs |
Get a collection of application/vnd.sas.datamgmt.job.status statuses of job executions for the given job flow. Authentication required: True |
Request:
GET http://www.example.com/SASDataMgmtBatchJob/jobFlowDefns/{id}/jobs
Response:
The response will contain a collection of application/vnd.sas.datamgmt.job.status items.
HTTP Response Codes: 200, 400, 401, 404, 500
POST /jobFlowDefns/{id}
Method | Description |
---|---|
POST /jobFlowDefns/{id} |
Execute the job flow, passing it the input data from the request. Authentication required: True |
Request:
POST http://www.example.com/SASDataMgmtBatchJob/rest/jobFlowDefns/{id}
The Content-Type
HTTP header of the request must be set to application/vnd.sas.datamgmt.jobflow.metadata+json. Otherwise, HTTP error 400 is returned.
JSON Representation Example of Request (application/vnd.sas.datamgmt.jobflow.metadata)
{
"inputs": {
"Process Priority": "low",
"Logging": "disable",
"dataTable": {
"metadata": [
{"name": "Full Name", "type": "string", "maxChars": 255},
{"name": "Age", "type": "integer"},
{"name": "Donor", "type": "boolean"},
{"name": "Birthday", "type": "date"},
{"name": "Temperature", "type": "real"}
],
"data": [
["sample string", 2, true, "2014-10-13", 5.5],
["another string", 246, false, "2014-12-31", 15.7075]
]
}
}
}
Response:
The response will contain a collection with a single application/vnd.sas.datamgmt.job.status item for the started job execution.
HTTP Response Codes: 200, 400, 401, 403, 404, 500
Resource /jobs
The /jobs
resource is a collection of statuses of jobs that are executed on the server. Pagination is not supported in the initial version of this resource.
GET /jobs
Method | Description |
---|---|
GET /jobs
|
List the statuses of all jobs that are executed on the server. Authentication required: True |
Request:
GET http://www.example.com/SASDataMgmtBatchJob/rest/jobs
Name | Type | Description |
---|---|---|
?name
|
string
|
List the statuses of all job executions with the specified URL-encoded job flow name. Note: This is a query for job executions of a single specific job flow. It is not a filter where a partial job flow name can be used or where multiple names can be specified. |
?user
|
string |
List the statuses of all job executions that are owned by the given URL-encoded user name. Note: This is a query for job executions by a single specific user. It is not a filter where a partial user name can be used or where multiple users can be specified. |
?state
|
string
|
List the statuses of all job executions in the given state (for example, Note: This is a query for job executions in a single specific state. Multiple states cannot be specified. |
Note: In this version of the API, combining queries or using multiple values for filters is not supported. For example, requests of this format will not work: 1) GET .../jobs?user=ram&state=failed.
2) GET .../jobs?user=ram|tyrath.
Response: The response will contain a collection of application/vnd.sas.datamgmt.job.status items.
JSON Representation Example (application/json)
{
"version": 1,
"name": "items",
"count": 1,
"items": [
{ "id": "MTQyMzg1OTc4OTo5OlBVQkxJQw==",
"jobFlowId": "MTAwRXhwLmRkZg==",
"name": "100Exp.ddf",
"startTimeStamp": "2015-02-13T15:37:44",
"stopTimeStamp": "2015-02-13T15:37:47",
"duration": 3,
"state": "completed",
"userName": "PUBLIC"
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobFlowDefns/MTAwRXhwLmRkZg==",
"method": "GET",
"rel": "jobFlow",
"type": "application/vnd.sas.datamgmt.jobflow.definition+json",
"uri": "/jobFlowDefns/MTAwRXhwLmRkZg=="
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs/MTQyMzg1OTc4OTo5OlBVQkxJQw==/log",
"method": "GET",
"rel": "log",
"type": "text/plain",
"uri": "/jobs/MTQyMzg1OTc4OTo5OlBVQkxJQw==/log"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs/MTQyMzg1OTc4OTo5OlBVQkxJQw==",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.collection+json",
"uri": "/jobs/MTQyMzg1OTc4OTo5OlBVQkxJQw=="
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs/MTQyMzg1OTc4OTo5OlBVQkxJQw==",
"method": "DELETE",
"rel": "delete",
"uri": "/jobs/MTQyMzg1OTc4OTo5OlBVQkxJQw=="
}
]
}
],
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/jobs",
"method": "GET",
"rel": "self",
"type": "application/vnd.sas.collection+json",
"uri": "/jobs"
},
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest",
"method": "GET",
"rel": "up",
"type": "application/vnd.sas.link+json",
"uri": "/"
}
]
}
HTTP Response Codes: 200, 401, 500
DELETE /jobs?state={jobState}
Method | Description |
---|---|
DELETE /jobs?state={jobState} |
Delete all jobs executed on the server that are in the given state. Authentication required: True |
Request:
DELETE http://www.example.com/SASDataMgmtBatchJob/rest/jobs?state={jobState}
Acceptable jobState
values are completed
, failed
, and canceled
. This is a query for job executions in a single specific state. Multiple states cannot be specified.
HTTP Response Codes: 204, 401, 403, 500
Resource /serverInfo
The /serverInfo
resource is a summary of select server configuration options and other information.
GET /serverInfo
Method | Description |
---|---|
GET /serverInfo |
List information about the server. Authentication required: True |
Request:
GET http://www.example.com/SASDataMgmtBatchJob/rest/serverInfo
Response:
The response will contain a set of name-value items.
{ "version": 1
"fullProductVersion": "Data Management Server version 2.6",
"fullVersion": "DataFlux Data Management Server version 9.6 (build 118)",
"majorProductVersion": 2,
"minorProductVersion": 6,
"productRevision": 0,
"majorVersion": 9,
"minorVersion": 6,
"buildNumber": 118,
"authenticationConnection": "",
"serverCurrentDate": "20141212T15:58:58",
"serverStartupDate": "20141212T15:29:16",
"minutesFromGmt": -300,
"databaseVersion": [
"Geo/Phone Database --- V3R2 --- 2013-06 --- expires: 3 months"
],
"libraryVersion": [
"Blue Fusion version 8.2 (build 162) --- blufsn82.dll",
"Verify Canada Extension version 1.5 (build 63) --- vcan15.dll",
"Blue Fusion Verify World version 1.5 (build 20) --- addverworld15.dll",
"AddressDoctor 4.1.16.478 --- AD_Validation_C.dll",
"Cluster Engine version 3.7 (build 29) --- ce37.dll"
],
"repositoryInfo": [
{ "name": "VDP_DB2Test_Repository",
"type": "Database",
"uuid": "528F3BF61ACD6E9C",
"version": 8
}
],
"links": [
{ "href": "http://www.sample.com/SASDataMgmtBatchJob/rest/",
"method": "GET",
"rel": "up",
"type": "application/vnd.sas.collection+json",
"uri": "/"
}
]
}
Resource /engines
The /engines
resource enables the user to delete all loaded batch jobs engines.
DELETE /engines
Method | Description |
---|---|
DELETE /engines |
Delete all loaded batch jobs engines. Authentication required: True |
Request:
DELETE http://www.example.com/SASDataMgmtBatchJob/rest/engines