---
title: "Complete Guide for Seven Bridges API R Client"
output:
  BiocStyle::html_document:
    toc: true
    toc_depth: 4
    number_sections: false
    highlight: haddock
    css: style.css
---


<!--
%\VignetteIndexEntry{Complete Guide for API R Client}
%\VignettePackage{sevenbridges}
%\VignetteEngine{knitr::rmarkdown}
-->


```{r include=FALSE}
knitr::opts_chunk$set(eval = FALSE)
```

## Introduction

This package is designed to support as many Seven Bridges supported platforms 
as possible, including the NCI Cancer Genomic Cloud Pilot developed by Seven Bridges, as long as you provide the correct API URL, it should work normally. 

Currently tested platform including

- [Cancer Genomics Cloud Platform](http://www.cancergenomicscloud.org/)
- [Seven Bridges US platform on AWS](https://www.sbgenomics.com/)
- [Seven Bridges US platform on Google Cloud](https://gcp.sbgenomics.com/)


It will be helpful to read complete platform documentation 

- [Full doc for Cancer Genomics Cloud Platform](http://docs.cancergenomicscloud.org/docs)
- [Full doc for Seven Bridges US platform](https://docs.sbgenomics.com/display/developerhub/Seven+Bridges+Genomics+Developer+Hub)

And it's API part if you want to understand details and use its full power:

- [API doc for Cancer Genomics Cloud Platform](http://docs.cancergenomicscloud.org/docs/the-cgc-api)
- [API doc for Seven Bridges US platform](https://docs.sbgenomics.com/display/developerhub/API)

### API URL

Most commonly used API url, choose the correct one depends on which platforms you are using


- __Seven Bridges US platform on AWS__ 
    1. V2 (new):  https://api.sbgenomics.com/v2/
    2. V1: https://api.sbgenomics.com/1.1/
- __Seven Bridges US platform on Google Cloud__ :  https://gcp-api.sbgenomics.com/v2/
- __Cancer Genomics Cloud Platform__ : https://cgc-api.sbgenomics.com/v2/

### API V1

The difference of V1 and V2 is that, V2 project is CWL(common workflow language) compatible, on NCI
cancer genomics cloud, google cloud, or any newer platforms, you should not have such a problem. But
for Seven Bridges Platofrm, currently users are able to create old project (V1) and new developer project (v2). For this group
of users, please note, `sevenbridges` package doesn't support old API at higher level, alterntatively
there are two ways if you are still using the old project type

- Use old `sbgr` package `sbgr` package will be under maintenance mode. 
- You can still use our lower api function as shown in following section, but higher level API 
calls like the chained action are not supported for v1. 

For advanced users, you can still use our lower API call for all V1 APIs like this, the most used 
arguments are "path", "query", "body", you can also check the function `api`. 

```{r}
library(sevenbridges)
a <- Auth(token = "8c3329a4de664c35bb657499bb2f335c", url = "https://api.sbgenomics.com/1.1/")
## Get projects: GET /1.1/project
a$api(path = "project")
## higher level doesn't work for v1
## a$project() won't work
```

Principle: I will recommend all users switch to new project type if possible, 
it will have more features and much better support from this package. 


### API V2 and beyond

Current public v2 API

- __NCI Cancer Genomics Cloud__ : https://cgc-api.sbgenomics.com/v2/
- __Seven Bridges on AWS__: https://api.sbgenomics.com/v2/
- __Seven Bridges on Google__: https://gcp-api.sbgenomics.com/v2/


`sevenbbridges` package only support V2 and later API, which support CWL compatible project. 

### Installation

This package is in devel branch on bioconductor now, to install the package.

```{r}
source("http://bioconductor.org/biocLite.R")
useDevel(devel = TRUE)
biocLite("sevenbridges")
```

If you cannot install it because you are not running latest R, you can install the package from github

```{r}
# install.packages("devtools") if devtools was not installed
## Install from github for development version  
source("http://bioconductor.org/biocLite.R")
# install.packages("devtools") if devtools was not installed
library(devtools)
install_github("sbg/sevenbridges-r", build_vignettes=TRUE, 
  repos=BiocInstaller::biocinstallRepos(),
  dependencies=TRUE)
```

This package is also available in docker image `tengfei/sevenbridges`, which the latest is build on `bioconductor/devel_base` on dockerhub.

Checkout he _Dockerfile_ at 

```{r}
system.file("docker", "sevenbridges/Dockerfile", package = "sevenbridges")
```



## Quickstart

For more details about how to use the API client in R, please go for the second section for complete guide. This section, I am going to use a simple example for a quick start.


### Create Auth object

Everything starts from an `Auth` object, so let's set up the `Auth` object, it remembers your auth token and url, every action started from this object. 

You have three different ways to setup the token. 

1. Direct setup via `Auth` function, explicityly setup token and url. 
2. Configuration file in your home folder ".sbg.auth.yml", very easy to manage and implicitly loaded, everytime you start a 
new R session.
3. Tempoerary setup via option in R session



```{r}
library(sevenbridges)
## direct setup
a <- Auth(token = "<fake_token>",
          url = "https://cgc-api.sbgenomics.com/v2/")
## or load default from config file (autoloaded into options)
library(sevenbridges)
a <- Auth(platform = "us", username = "tengfei")
```

```
== Auth ==
token : <fake_token>
url : https://cgc-api.sbgenomics.com/v2/
```

### Infomation about a user

This call returns information about your account.

```{r}
a$user()
```

```
== User ==
href : https://cgc-api.sbgenomics.com/v2/users/tengfei
username : tengfei
email : tengfei.yin@sbgenomics.com
first_name : Tengfei
last_name : Yin
affiliation : Seven Bridges Genomics
country : United States
```

To list user resrouces, This call returns information about the specified user. Note that currently you can view only your own user information, and so this call is equivalent to the call to Get my information.

```{r}
a$user(username = "tengfei")
```

### Rate limit

This call returns information about your current rate limit. This is the number of API calls you can make in one hour.

```{r}
a$rate_limit()
```

```
== Rate Limit ==
limit : 1000
remaining : 993
reset : 1457980957
```

### Show billing information

Billing information, every project is associated with a billing group

```{r}
## check your billing info
a$billing()
a$invoice()
```

For more information, use `breakdown = TRUE`

```{r}
a$billing(id = "your_billing_id", breakdown = TRUE)
```


### Create a project

Create a new project called "api testing", with the billing group id. 

```{r}
## get billing group id
bid <- a$billing()$id
## create new project
(p <- a$project_new(name = "api testing", bid, description = "Just a testing"))
```

```
== Project ==
id : tengfei/api-testing
name : api testing
description : Just a testing
billing_group_id : <fake_bid>
type : v2
-- Permission --
```

### Import CWL app and run a task
Then we want to run a task, and you want to upload your json file that describe your tool, here we use an random number generator example saved in this package. 

Note: alternatively you can directly describe your CWL tool in R with this package, please read another vignettes on "Describe CWL Tools/Workflows in R and Execution"

```{r, eval = TRUE}
## Add an CWL file to your project
fl.runif <- system.file("docker", "sevenbridges/rabix/runif.json", package = "sevenbridges")
```

Let's take a look at the content of this json file

```{r comment='', eval = TRUE}
cat(readLines(fl.runif), sep = '\n')
```

Now let's add this json file to your projct as a new app.

```{r}
## name your app 
(aid <- p$app_add("runif", fl.runif)$id)
```

You get an id like this

```
"tengfei/api-testing/runif/0"
```

It's composed of 

1. __project id__ : tengfei/api
2. __app short name__ : runif 
3. __revision__ : 0


Alternatively, you can describe tools like this

```{r comment='', eval = TRUE}
fl <- system.file("docker", "sevenbridges/rabix/generator.R", package = "sevenbridges")
cat(readLines(fl), sep = '\n')
```

And add it like this

```{r}
## rbx is the object returned by Tool function
(aid <- p$app_add("runif", rbx)$id)
```

Please read another tutorial about how to describe tools and flows in R. 

### Excute a new task

Now we draft a new task, you need to specify

- name of the task
- description
- app id
- inputs of your task: you need to know about the app you are running, in this case, the cwl app accept 4 parameters (number, min, max, seed). 


I want to generate 1 random number (default) between 1 and 10. 

```{r}
(tsk <- p$task_add(name = "randome number 5", 
           description = "Generate some random number", 
           app = aid,
           inputs = list(min = 1, max = 10, number = 1, seed = 4)))

## confirm, show all task status is draft
p$task(status = "draft")

## or just list the task
tsk$update()
```

Now you have your draft task, you can delete your draft task or update it.

```{r}
## not run
## tsk$delete()
```

looks like there is only a single number, oops, I want 100 numbers instead, so 
let me update my draft task 

```{r}
tsk$getInputs()
## missing number input, only update number
tsk$update(inputs = list(number = 500, seed = 1))
## double check 
tsk$getInputs()
```

Or we just want to run it in the cloud!

```{r}
## Run your task
tsk$run()
```
To monitor the task, you can always call `update` on task object to check the status. 

```{r}
tsk$update()
```

Or more fun, you can monitor a running task with hook function, so trigger a function when that status is "completed", "running" etc, please check the 
details in section about hook of task. 

By default it just show message when the task is completed. 

```{r}
## Monitor your task (skip this part)
## tsk$monitor()
```

To about running task just  all

```
## not run
## tsk$abort()
```

To download all files from a completed tasks

```{r}
tsk$download("~/Downloads")
```

More fun to set task hook, so when it's complete download the files

```{r}
setTaskHook("completed", function(){
    tsk$download("~/Downloads")
})
tsk$monitor()
```


## User-friendly API

This is what the package try to help, and provide a user-friendly interface that we suggest our users to use, so you don't have to combine several `api()` calls and refer to the API documentation all the times to finish a simple task. 


### Authentification

#### Set up default token for different platforms

You can create a file called '.sbg.auth.yml' in your home folder, and maintain multiple account for a list of platforms, including private or public ones. 

```
us:
  url: https://api.sbgenomics.com/v2/
  user:
    tengfei:
      token: fake_token
    yintengfei:
      token: fake_token
cgc:
  url: https://cgc-api.sbgenomics.com/
  user:
    tengfei:
      token: fake_token
gcp:
  url: https://gcp-api.sbgenomics.com/v2/
  user:
    tengfei:
      token: fake_token
```

When you load sevenbridges package, it will first try to parse your token configuration file first into an options list. 

```{r}
## Create Auth object from config file
a <- Auth(username = "yintengfei", platform = "us")
## show all 
getToken()
## show all pre-set user token for platform 
getToken("cgc")
## show individual token for a user
getToken(platform = "cgc", username = "tengfei")
```

Note: when you edit your .sbg.auth.yml, you have to reload your package. 


#### Create Auth object directly

First thing first, you need to construct an Auth object, everything begins with
this object, it stores

- The authentication token
- The API URL
- The platform (US platform, Cancer Genomics Cloud etc), this is optional, will translate into API url.

The logic is like this

1. If you didn't pass url or token, we think you are loading from config file
2. if no platform or user provided, will use the first item in your token config file, __this is not recommended, at least provide platform/username set.__ 


```{r}
library(sevenbridges)
## direct setup
a <- Auth(token = "1c0e6e202b544030870ccc147092c257",
          url = "https://cgc-api.sbgenomics.com/v2/")
```

By default it points to Cancer Genomics Cloud platform, unless you specify

- API URL (more flexible)
- or Platform (currently support 'cgc', 'us', 'gcp')


Note: when you construct the Auth object, make sure you input the correct platform or API url for your authentication. On Seven Bridges related platforms, 
you can always find it under your account setting and developer tab. 

For the tutorial about how to get your authentication, please check

- [Get token from US platform](https://docs.sbgenomics.com/display/developerhub/Authentication+Token)
- [Get token from Cancer Genomics Cloud Platform](http://docs.cancergenomicscloud.org/docs/the-cgc-api)

### List All API calls

If we didn't pass any parameters to api() from Auth, it will list all API calls,
and anything parameter we provided will pass on to api() function, but you don't 
need to input token and url again! The Auth object will know that information already.\
And this call from Auth object will check the response too. 

```{r}
a$api()
```

### Offset and Limit and Search 

`offset` specify where it is started, and `limit` specify how many you want to show from there (max: 100). Because the item could be thousands of files and apps, so by default the offset and limit is set to 0 and 100 accordingly.

```{r}
getOption("sevenbridges")$offset
getOption("sevenbridges")$limit
```

Please pay attention to this

- Search by `id` is most accurate and fast for any Item like Project, App, Task, File. 
- Search by name will only search across current pulled content, so use `complete = TRUE` if you want to search across everything, this might be slow. 

For example, to list all public apps, use `visibility` argument, but make sure you pass `complete = TRUE` to it, to show every single things. This arguments generally works for items like "App", "Project", "Task", "File" etc 

```{r}
## first, search by id is fast
x <- a$app(visibility = "public", id = "djordje_klisic/public-apps-by-seven-bridges/sbg-ucsc-b37-bed-converter/0")

## show 100 items from public
x <-  a$app(visibility = "public")
length(x) ## 100


x <- a$app(visibility = "public", complete = TRUE)
length(x) ## 211 by March, 2016
## this return nothing, because it's not in the first 100 
a$app(visibility = "public", name = "bed converter")
## this return an app, because it pulls all apps and did serach.
a$app(visibility = "public", name = "bed converter", complete = TRUE)
```


### Rate Limits

This call returns information about your current rate limit. 
This is the number of API calls you can make in one hour.

```{r}
a$rate_limit()
```

### Users

This call returns a list of the resources, such as projects, billing groups, and 
organizations, that are accessible to you. If you are not an administrator, this call will only return a successful response if {username} is replaced with your own username. If you are an administrator, you can replace {username} with the username of any CGC user, to return information on their resources.

_Case sensitivity_: Don't forget to capitalize your username in the same way as you set it when you registered on the CGC.

If you don't provide a username, your user information will be shown.

```{r}
## return your information
a$user()
## return user 'tengfei''s information
a$user("tengfei")
```


### Billing Group and Invoices

#### For billing

if no id provided, This call returns a list of paths used to access billing information via the API. else, This call lists all your billing groups, including groups that are pending or have been disabled. if `breakdown = TRUE`, This call returns a breakdown of spending per-project for the billing group specified by billing_group. For each project that the billing group is associated with, information is shown on the tasks run, including their initiating user (the runner), start and end times, and cost.


```{r}
## return a BillingList object
(b <- a$billing())
a$billing(id = b$id, breakdown = TRUE)
```

#### For invoices

If no id provided, This call returns a list of invoices, with information about each, including whether or not the invoice is pending and the billing period it covers. The call returns information about all your available invoices, unless you use the query parameter bg_id to specify the ID of a particular billing group, in which case it will return the invoice incurred by that billing group only. if id provided, This call retrieves information about a selected invoice, including the costs for analysis and storage, and the invoice period.

```{r}
a$invoice()
a$invoice(id = "fake_id")
```


Note (TODO): Invoice is not an object yet, it currently just return a list.



### Project Operation

Project is the basic unit to organize different entities: files, tasks, apps, etc. So lots actions comes from this `Project' object. 

#### List All Projects 

This call returns a list of all projects you are a member of. Each project's project_id and URL on the CGC will be returned.

```{r}
a$project()
```

Then if you want to list the projects owned by and accessible to a particular user, specify the `owner` argument. Each project's ID and URL will be returned.

```{r}
a$project(owner = "tengfei")
a$project(owner = "yintengfei")
```

To get details about project(s), use `detail = TRUE`

```{r}
a$project(detail = TRUE)
```

#### Partial Match Project Name

For more friendly interface and convenient search, we support partial name match in this interface. The first argument for the call is "name", 
users can provide part of the name and we do a search for you automatically. 

```{r}
## want to return a project called
a$project("hello")
```

#### Create a New Project

To create a new project, user need to specify

- name (required)
- billing_group_id (required)
- description (optional)
- tags (optional): this has to be a list(), only if you are "TCGA" user, you can 
create TCGA project by passing tags list("tcga")
- type (optional): by default, we are creating a cwl project "v2"



```{r}
a$project_new("api_testing_tcga", b$id,
              description = "Test for API")
```

#### Create a new project with TCGA controlled data on CGC

Just need to pass a "tags" list with value "tcga"


```{r}
a$project_new("controlled_project", b$id,
              description = "Test for API", tags = list("tcga"))
```

#### Delete a Project

Next we delete what we created for testing, only *single* project could be deleted now by call `$delete()`, so
please pay attention to the returned object from `a$project()`, sometimes if you are using partial matching by name, it will return a list. If you want to operate on a list of object, we provide some batch function, please read relevant section.

```{r}
## remove it, not run
a$project("api_testing")$delete()
## check
## will delete all projects matcht the name
delete(a$project("api_testing_donnot_delete_me"))
```

#### Update/Edit a Project

You can update information about an existing project, including

- name
- description
- billing_group

```{r}
a$project(id = "tengfei/helloworld")
a$project(id = "tengfei/helloworld")$update(name = "Hello World Update", 
                                                    description = "Update description")

```

#### Project Member

##### List members

This call returns a list of the members of the specified project. For each member, the response lists:

- The member's username on the CGC
- The member's permissions in the project specified

```{r}
a$project(id = "tengfei/demo-project")$member()
```

##### Add a member
This call adds a new user to a specified project. It can only be successfully made by a user who has admin permissions in the project.

Requests to add a project member must include the key permissions. However, if you do not include a value for some permission, it will be set to false by default.

Set permission by passing: copy, write, execute, admin, read argument. 

Note: read is implicit and set by default, you can not be project member without having read permission

```{r}
m <- a$project(id = "tengfei/demo-project")$member_add(username = "yintengfei")
```

#### Update a member

This call edits a user's permissions in a specified project. It can only be successfully made by a user who has admin permissions in the project.

```{r}
m <- a$project(id = "tengfei/demo-project")$
       member(username = "yintengfei")
m$update(copy = TRUE)
```

```
== Member ==
username : yintengfei
-- Permission --
read : TRUE
write : FALSE
copy_permission : TRUE
execute : FALSE
admin : FALSE
```

##### Delete a member

To delete an existing member, just to call `delete()` action on `Member` object.

```{r}
m$delete()
## confirm
a$project(id = "tengfei/demo-project")$member()
```

#### List all Files

To list all files belongs to a project simple use

```{r}
p <- a$project(id = "tengfei/demo-project")
p$file()
```
### Files and Metadata

#### List all files

This call returns a list of all files in a specified project that you can access. For each file, the call returns:

- Its ID
- Its filename

The project is specified as a query parameter in the call.

```{r}
a$file(project = p$id)
a$file("omni", project = p$id, detail = TRUE)
```

However we recommend user use cascading way to list files. 

```{r}
p$file()
```

To get details about files, please use `detail = TRUE` in the call.

```{r}
## need to check
p$file(detail = TRUE)
```

#### Copy a file or group of files

This call copies the specified file to a new project. Files retain their metadata when copied, but may be assigned new names in their target project.

Note that Controlled Data files may not be copied to Open Data projects. To make this call, you should have copy permission within the project you are copying from.

Let's try to copy a file from CGC public files, the id you can tell from the url is 
"561e1b33e4b0aa6ec48167d7"

You must provide 

- `id` file id, or list/vector of files ids. 
- `project` parameter: project id. 
- `name` is optional, if omitted, use the same.

```{r}
## 1000G_omni2.5.b37.vcf
fid <- "561e1b33e4b0aa6ec48167d7"
fid2 <- "561e1b33e4b0aa6ec48167d3"
pid <- a$project("demo")$id
a$copyFile(c(fid, fid2), project = pid)
a$project(id = pid)$file()
```

NOTE: to copy a group of files, you need `Auth$copyFile()` interface. The id of those files in your project will be different from public id. 

Alternatively you can do __single file__ copy like this

```{r}
a$project("hello")$file(id = fid)$copyTo(pid)
```

#### Delete file(s)

Note: the `delete` action only work for single file now, make sure your `file` call return 
a single file not a file list.

```{r}
a$project("demo")$file()[[1]]$delete()
## confirm the deletion
a$project("demo")$file()
```

You can also delete a group of files or `FilesList` object, __be careful__ with this function! 

```{r}
## return 5 files
a$project("demo")$file("phase1")
## delete all of them
delete(a$project("demo")$file("phase1"))
a$project("demo")$file("phase1")
```

#### Download files

To get the download information, basically a url, please use

```{r}
a$project("demo")$file()[[1]]$download_url()
```

To download directly from R, use `download` call directly from single File object.

```{r}
fid <- a$project("demo")$file()[[1]]$id
a$project("demo")$file(id = fid3)$download("~/Downloads/")
```

I also created `download` function for `FilesList` object to save your time

```{r}
fls <- a$project("demo")$file()
download(fls, "~/Downloads/")
```

To download all files from a project.

```{r}
a$project("demo")$download("~/Downloads")
```

#### Upload files

Seven Bridges platforms provide couple different ways for data import

- command line uploader
- graphic UI uploader
- from ftp, http etc from interface directly
- api uploader that you can directly call with sevenbridges package

API client uploader is working like this, simply call `project$upload` function to 
upload a file a file list or a folder recursively...

```{r}
a <- Auth(username = "tengfei", platform = "cgc")
fl <- system.file("extdata", "sample1.fastq", package = "sevenbridges")

(p <- a$project(id = "tengfei/quickstart"))

## by default load .meta for the file
p$upload(fl, overwrite = TRUE)
## pass metadata
p$upload(fl, overwrite = TRUE, metadata = list(library_id = "testid2", platform = "Illumina x11"))
## rename
p$upload(fl, overwrite = TRUE, name = "sample_new_name.fastq", 
         metadata = list(library_id = "new_id"))


```

Upload a folder

```{r}
dir.ext <- system.file("extdata", package = "sevenbridges")
list.files(dir.ext)
p$upload(dir.ext, overwrite = TRUE)
```

Upload a file list


```{r}
dir.ext <- system.file("extdata", package = "sevenbridges")
## enable full name
fls <- list.files(dir.ext, recursive = TRUE, full.names = TRUE)
p$upload(fls, overwrite = TRUE)
p$upload("~/Documents/Data/sbgtest/1000G_phase1.snps.high_confidence.b37.vcf")
```
#### Files filter from a task

You can also get all files from a task, or by a metadata filter. 


```{r}
## list all outputs file from a task id
a$task(id = "53020538-6936-422f-80de-02fa65ae4b39")$file()

## alternative way to list files under specific project
a$file(project = "tengfei/re",
       origin.task = "53020538-6936-422f-80de-02fa65ae4b39")


## you can filter by metadata as well
a$file(project = "tengfei/re", 
       origin.task = "53020538-6936-422f-80de-02fa65ae4b39", 
       metadata = list(experimental_strategy = "RNA-Seq"))
```




#### Public files

The only way you can list public files now via API is from `Auth$file()` call, and for now, 
the project id is "admin/sbg-public-data". This may be updated later. Alternative, just click the file from our GUI, you will see the id in the url link.

```{r}
a$file(project = "admin/sbg-public-data")
```

#### Update a file

You can call `update()` function from Files object, following things could be updated

- name
- metadata (list): this is going to overwrite all meta for the file, so please provide the full list. For more flexible operation, please check next section about Metadata. 

If no parameters provided, will just get detail for the same file and update the object
itself. 

```{r}
(fl <- a$project(id = "tengfei/demo-project")$file(name = "sample.fastq"))
```

```
== File ==
id : 56c7916ae4b03b56a7d7
name : sample.fastq
project : tengfei/demo-project
```

Show metadata

```{r}
## show metadata
fl$meta()
```

Update meta

```{r}
fl$update(name = "sample.fastq", metadata = list(new_item1 = "item1", new_item2 = "item2", file_extension = "fastq"))
## check it out
fl$meta()
```


#### Metadata Operation

A full list of metadata fields and their permissible values on the CGC is available on the page [TCGA Metadata](http://docs.cancergenomicscloud.org/v1.0/docs/tcga-metadata-on-the-cgc).
 
Note that the file name is not the same as its ID. The ID is a hexadecimal string, automatically assigned to a file in a project. The file's name is a human-readable string. For information, please see the API overview.


To get metadata for a file call `meta()`. 

```{r}
## meta is pulling the latest information via API
fl$meta()
## field meta data saved the previous saved one
fl$metadata
```


Although CGC defined a set of meta schema, which is visible on the UI of the platform, but you can pass any free form of meta for the file, it's just not visible on UI, but it's stored with the data. 

Only the value being specified stored with files, to set metadata please call `setMeta()` from
Files object. 

__Important__:

- By default, we are not overwriting the meta field using `setMeta` call, unless you pass
the `overwrite = TRUE` argument

```{r}
fl$setMeta(new_item3 = "item3")
fl
## oops it removed rest of the meta
fl$setMeta(new_item4 = "item4", overwrite = TRUE)
fl
```

Let's keep playing with meta, if you are really interested in the default schema that shown on the UI, you can use `Metadata()` constructor and check details of each meta; Simply call the function (name of meta), it will show description and enumerated items. Please pay attention to `suggested_values` field.

```{r}
## check which schema we have
Metadata()$show(full = TRUE)
## check details for each, play with it
platform()
paired_end()
quality_scale()
```

You can see some have suggested value, to construct the Metadata, we encourage you use `Metadata()` directly, pass metadata directly into the call, it will do the validation. 

```{r}
Metadata(platform  = "Affymetrix SNP Array 6.0", 
         paired_end = 1, 
         quality_scale = "sanger",
         new_item = "new test")
```

### App

From now on we are going to have fun with Apps! The CWL(Common Workflow Language) based approach. It gets more and more popular and really designed for reproducible pipeline description and execution. All Seven Bridges platforms support cwl naively in the cloud. 
So in this section, I will introduce how we are going to do this via API and inside R. 

#### List all apps 

This call lists all the apps available to you.

```{r}
a$app()
## or show details
a$app(detail = TRUE)
```

To search a name, please pass a pattern for the `name` argument; or provide a unique `id`.

```{r}
## pattern match
a$app(name = "STAR")
## unique id
aid <- a$app()[[1]]$id
aid
a$app(id = aid)
## get a specific revision from an app
a$app(id = aid, revision = 0)
``` 



To list all apps belong to one project use `project` argument

```{r}
## my favorite, always
a$project("demo")$app()

## or alternatviely
pid <- a$project("demo")$id
a$app(project = pid)
```

To list all public apps, use `visibility` argument

```{r}
## show 100 items from public
x = a$app(visibility = "public")
length(x)
x = a$app(visibility = "public", complete = TRUE)
length(x)
x = a$app(project = "tengfei/helloworld", complete = TRUE)
length(x)
a$app(visibility = "public", limit = 5, offset = 150)
```

To search an app cross all published apps (this may take a while)


```{r}
a$app("STAR", visibility = "public", complete = TRUE)
```

#### Copy an App

This call copies the specified app to the specified project. The app should be one in a project that you can access; this could be an app that has been uploaded to the CGC by a project member, or a publicly available app that has been copied to the project.

Need two arguments

- project: id character
- name: optional, to re-name your app

```{r}
aid <- a$app(visibility = "public")[[1]]$id
a$copyApp(aid, project = pid, name = "copy-rename-test")
## check it is copied
a$app(project = pid)
```

#### Get CWL from an App

This call returns information about the specified app, as raw CWL. The call differs from the call to GET details of an app by returning a JSON object that is the CWL.

The app should be one in a project that you can access; this could be an app that has been uploaded to the CGC by a project member, or a publicly available app that has been copied to the project.

To get a specific revision, pass `revision` argument. 

```{r}
ap <- a$app(visibility = "public")[[1]]
a$project("demo")$app("index")
## get a specific revision
a$project("demo")$app("index", revision = 0)

```

TODO: convert it to an CWL object

#### Add CWL as an APP

Use `app_add` function call from a `Project` object, two parameters required

- short_name: a short id for your app, alphanumeric character, no spacing; this not name field. 
- filename: you json file for cwl.

```{r}
cwl.fl <- system.file("extdata", "bam_index.json", package = "sevenbridges")
a$project("demo")$app_add(short_name = "new_bam_index_app", filename = cwl.fl)
a$project("demo")$app_add(short_name = "new_bam_index_app", revision = 2, filename = cwl.fl)
```

Note: provide the same short_name will add new revision

#### Directly Describe CWL in R

This is fun and is introduced in another vignette. 


### Task Operation

#### List tasks

This call returns a list of tasks that you can access. You are able to filter by status

```{r}
## all tasks
a$task()
## filter
a$task(status = "completed")
a$task(status = "running")
```

To list all tasks in a project

```{r}
## better way
a$project("demo")$task()

## alternatively
pid <- a$project("demo")$id
pid
a$task(project = pid)
```

#### Create a draft task

To create a draft, you need to call the `task_add` function from Project object. And you need to pass following arguments

- name: name for this task
- description: description for this task
- app: app id you have access to
- inputs: inputs list for this task

```{r}
## push an app first
fl.runif <- system.file("extdata", "runif.json", package = "sbgr")
a$project("demo")$app_add("runif_draft", fl.runif)
runif_id <- "tengfei/demo-project/runif_draft"
## create a draft task
a$project("demo")$task_add(name = "Draft runif 3", 
                           description = "Description for runif 3", 
                           app = runif_id,
                           inputs = list(min = 1, max = 10))
## confirm
a$project("demo")$task(status = "draft")
```

#### Modify a task

Call `update` function fro a Task object, you can update

- name
- description
- inputs list (only update items you provided.)

```{r}
## get the single task you want to update
tsk <- a$project("demo")$task("Draft runif 3")
tsk
tsk$update(name = "Draft runif update", description = "draft 2", 
           inputs = list(max = 100))
## alternative way to check all inputs
tsk$getInputs()
```

#### Run a task

This call runs (executes) the specified task. Only tasks whose status is "DRAFT" may be run.

```{r}
tsk$run()
## run update without information just return latest information
tsk$update()
```

#### Monitor a running task and set function hook

To monitor a running task, call `monitor` from a task object

- first argument set interval time to check the status
- rest arguments might be used for hook function



```{r}
tsk$monitor()
```

get and set default hook function for task status, currently failed, completed tasks
will break the monitoring. 

Note: Hook function has to return `TRUE` (break monitoring) or `FALSE` (continuing) in the end. 

```{r}
getTaskHook("completed")
getTaskHook("draft")
setTaskHook("draft", function(){message("never happens"); return(TRUE)})
getTaskHook("draft")
```

#### Abort a runing task

This call aborts the specified task. Only tasks whose status is "RUNNING" may be aborted.

```{r}
## abort
tsk$abort()
## check
tsk$update()
```



#### Delete a task

Note that you can only delete draft tasks, not running tasks.

```{r}
tsklst <- a$task(status = "draft")
## delete a single task
tsklst[[1]]$delete()
## confirm
a$task(status = "draft")
## delete a list of tasks
delete(tsklst)
```

#### Download all files from a completed task

```{r}
tsk$download("~/Downloads")
```

#### Run task in bacth mode

To run task in batch mode, (check `?batch`) for more details, here is an mock running

```{r}
## batch by items
(tsk <- p$task_add(name = "RNA DE report new batch 2", 
                   description = "RNA DE analysis report", 
                   app = rna.app$id,
                   batch = batch(input = "bamfiles"),
                   inputs = list(bamfiles = bamfiles.in, 
                                 design = design.in,
                                 gtffile = gtf.in)))

## batch by metadata, input files has to have metadata fields specified
(tsk <- p$task_add(name = "RNA DE report new batch 3", 
                   description = "RNA DE analysis report", 
                   app = rna.app$id,
                   batch = batch(input = "fastq", 
                                 c("metadata.sample_id", "metadata.library_id")),
                   inputs = list(bamfiles = bamfiles.in, 
                                 design = design.in,
                                 gtffile = gtf.in)))
```

### Get Raw Response from httr

In easy API, we return an object which contains the raw response from httr as a field,
you can either call `response()` on that object or just get the field out of it

### Batch operation on project/files/tasks

Right now,  users have to use `lapply` to do those operations themselves. It's simple implementation. 

In this package, we implement `delete` and `download` for some object like task and project or file. 

## Cheatsheet

Quick cheat sheet (in progress)

```{r}
## Authentification
getToken()
a <- Auth(token = token)
a <- Auth(token = token, 
          url = "https://cgc-api.sbgenomics.com/v2/")
a <- Auth(platform = "us", username = "tengfei")

## list API
a$api()

## Rate limits
a$rate_limit()

## Users
a$user()
a$user("tengfei")

## billing
a$billing()
a$billing(id = , breakdown = TRUE)
a$invoice()
a$invoice(id = "fake_id")

## Project
### create new project
a$project_new(name = , billing_group_id = , description = )
### list all project owned by you
a$project()
a$project(owner = "yintengfei")
### partial match
p <-  a$project(name = , id = , exact = TRUE)
### delete
p$delete()
### update
p$update(name = , description = )
### members
p$member()
p$member_add(username = )
p$member(username = )$update(write = , copy = , execute = )
p$memeber(usrname = )$delete()

## file
### list all files in this project
p$file()
### list all public files
a$file(visibility = "public")
### copy
a$copyFile(c(fid, fid2), project = pid)
### delete
p$file(id = fid)$delete()
### download
p$file()[[1]]$download_url()
p$file(id = fid3)$download("~/Downloads/")
### download all
download(p$file())
### update a file
fl$update(name = , metadata = list(a =  ,b = , ...))
### meta
fl$meta()
fl$setMeta()
fl$setMeta(..., overwrite = TRUE)

## App
a$app()
### apps in a project
p$app()
p$app(name, id, revision = )
a$copyApp(aid, project = pid, name = )
### add
p$app_add(short_name = , filename =)

## Task
a$task()
a$task(name = , id = )
a$task(status = )

p$task()
p$task(name = , id = )
p$task(status = )

tsk <- p$task(name = , id = )
tsk$update()
tsk$abort()
tsk$run()
tsk$download()
tsk$detele()
tsk$getInputs()
tsk$monitor()

getTaskHook()
setTaskHook(statis = , fun  =)
```