File Mover
File Mover handles high volumes of massive file transfers from A to B quickly and efficiently so that you don't have to worry about the complicated and minute details associated with storage providers, TCP/HTTP protocols, or the headaches dealing with transient errors and dropped packets.
You give it a source to fetch data from and a destination to send the data to, then it goes to work. If the source or destination need extra headers you can pass them along with the request. Optionally, you can provide webhook details in the request and it will let you know when it's done.
All details of your request are encrypted in transit and at rest. Regardless, we recommend using short lived, presigned URLs and temporary tokens where possible.
An example request
Here's a typical example for transferring a file from a URL to an S3 bucket:
File Mover request
const response = await fetch(
"https://cakeslice.com/api/v1/file_mover",
{
method: "POST",
headers: {
"Authorization": `Bearer ${token}`,
"Content-Type": "application/json"
},
body: JSON.stringify({
source: {
type: "url",
url: "https://www.nasa.gov/sites/default/files/thumbnails/image/curiosity_selfie.jpg",
},
destination: {
type: "s3",
bucket_name: "example_bucket",
file_name: "curiosity_selfie.jpg"
}
})
}
)
await response.json()
Response
The request payload
The request is made up of three parts, two of which are required (source and destination) and one which is optional (webhook).
- Name
- source
- Type
- required object
- Description
The location of the file to fetch.
- Name
- destination
- Type
- required object
- Description
The location to send the file to.
- Name
- webhook
- Type
- optional object
- Description
Where to call back with results upon completion.
File Mover request
const response = await fetch(
"https://cakeslice.com/api/v1/file_mover",
{
method: "POST",
headers: {
"Authorization": `Bearer ${token}`,
"Content-Type": "application/json"
},
body: JSON.stringify({
source: {
// source params here
},
destination: {
// destination params here
},
webhook: {
// webhook params here
}
})
}
)
await response.json()
Source and destination params
The source and destination params can be any combination of a given type, and the attributes required depend on which type is given.
URL type:
- Name
- type
- Type
- required string
- Description
Needs to be "url".
- Name
- url
- Type
- required string
- Description
The URL to stream data from / to.
- Name
- verb
- Type
- optional string
- Description
Valid values are "GET", "PUT", "POST", "PATCH". Defaults to "GET".
- Name
- headers
- Type
- optional object
- Description
Headers to forward along with the request.
URL source / destination params
{
"type": "url",
"url": "https://example.com/file.mp4",
"verb": "GET",
"headers": {
"Authorization": "Bearer abcd.1234.jwt"
}
}
S3 and GCS types:
- Name
- type
- Type
- required string
- Description
Valid values are "s3" and "gcs".
- Name
- bucket_name
- Type
- required string
- Description
The bucket the data exists in or will be uploaded to.
- Name
- file_name
- Type
- required string
- Description
The name of the file in the bucket to download or upload.
- Name
- auth_token
- Type
- optional string
- Description
Used for authenticating against private buckets and objects.
S3 source / destination params
{
"type": "s3",
"bucket_name": "videos",
"file_name": "file.mp4",
"auth_token": null
}
The webhook param
If provided, File Mover will make a POST request to the given URL upon completion with the results of the operation. Learn more about webhook requireds here.
- Name
- url
- Type
- required string
- Description
The URL you want File Mover to send its results to.
- Name
- headers
- Type
- optional object
- Description
Headers to forward along with the request.
Webhook params
{
"url": "https://example.com/webhook",
"headers": {
"X-Custom-Header": "Results from Cake Slice"
}
}