Working with Nested JSON & R

Working with JSON objects in R can be confusing. This model aims to show how JSON is parsed coming to and leaving from a ScienceOps model.

For simplicity, we'll have this model do 2 things:

  1. Add a random number after the users name
  2. Restructure the response to return JSON arrays for each user.

Below is the JSON that we'll be sending in to our request.

test_data <- [{
    "users": [{
        "ts": "Thu Mar 25 03:13:01 UTC 2010",
        "user": {
            "name": "Lamur",
            "user_id": "68694999"
        }
    }, {
        "ts": "Thu Mar 25 03:13:08 UTC 2010",
        "user": {
            "name": "Sam",
            "user_id": "686943299"
        }
    }],
    "lastVote": {
        "timestamp": 1269486788526,
        "user": {
            "name": "Lamur",
            "user_id": "686923499"
        }
    }
}]

Expected Model Response:

[
    ["Lamur 1", "68694999", "Thu Mar 25 03:13:01 UTC 2010"],
    ["Sam 2", "686943299", "Thu Mar 25 03:13:08 UTC 2010"]
]

When ScienceOps receives data for a prediction, it parses it. This ensures it's received a valid request.

For R models, this is done with the package rjson. This means that prior to the model.predict function being called, the data is parsed and transformed into an R object.

This means that an R object is actually being passed to the model.predict function

Generally, think of it as below: API request --> rjson::fromJSON(data) ---> model.predict(data) ---> rjson::toJSON()--> response

# below (for examples sake) is incoming JSON parsed BEFORE it gets sent to model.predict()
#API Requests --> model.predict(rjson::fromJSON(df))

model.predict <- function(df) {
  lg <- df
  m <- lapply(lg[[1]]$users, function(x) c(paste(x$user['name'],sample(1:2,1)), x$user['user_id'], x['ts']))
  m <- do.call(rbind, m)
  # below, we split the dataframe so it is returned as JSON arrays
  m <- unname(split(m, 1:nrow(m)))
  m
}

# rjson::toJSON() --> API Response

Whats a little tricky here is testing our model.predict. If we want to test it locally with JSON data, we need to do the parsing ourselves.

Below shows how we can do that:

# Use rjson to parse the incoming JSON
test_input <- rjson::fromJSON(test_data)

# Use rjson to parse back to JSON
rjson::toJSON(model.predict(test_input))

#RESULTS (ugly but just wait!)
[1] "[[\"Lamur 2\",\"68694999\",\"Thu Mar 25 03:13:01 UTC 2010\"],[\"Sam 2\",\"686943299\",\"Thu Mar 25 03:13:08 UTC 2010\"]]"

Now that we've achieved the JSON structure, lets deploy and test the model on ScienceOps:

yhat.config  <- c(
  username="kermit",
  apikey="123456789abcdefg",
  env="https://sandbox.yhathq.com/"
)
yhat.deploy("JSON_parsing")

On ScienceOps, we can test on the Scoring page and see the JSON is formatted as expected.

results matching ""

    No results matching ""