AutoML Tables forecasting model

[Maayan Harel]

Hi,

I'm trying to train a simple test run with automl forecasting tables. 

After loading the data and defining the model (the series identifier, target, timestamp column ...), I get the error:

"Unable to start training due to the following error: Internal error encountered."

when I try to start training (a sec after I try starting).

The data for the model is very simple (test run), just a single covariate (other than the target).

What could cause this error?

Thanks,

Maayan

[Helin Wang]

+Joanne Qi our oncall to help take a look. Thanks Joanne!

--
You received this message because you are subscribed to the Google Groups "cloud-automl-tables-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-automl-tables...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-automl-tables-discuss/8a6b3af4-67c6-4f4d-8e6c-959afa9d23f5n%40googlegroups.com.

[Joanne Qi]

Hi Mayaan, 

Could I get more information about your training inputs? Are you using CSV/BigQuery? What is the schema of your BigQuery table if you are using it? Did you set any transformations? If possible, could I get your project ID/number? Thanks! 

[Maayan Harel]

Hi,

I'm using CSV input, very simple just a numeric covariate column, a Series identifier set to const (as I understand it is needed),  a Time, and a target. The data is Numeric, except a Time column which is Timestamp type. See fig below. 

Is there somewhere I can send the csv file? can't add attachments to this chat (other than a photo))

Thanks!

Maayan

[Dawei Jia]

We need your project ID and training job id to investigate this failure.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-automl-tables-discuss/725a7c3f-ccdf-42c3-b545-2a3c9859360dn%40googlegroups.com.

[Maayan Harel]

Hi,

replied in email with project details. There is no training job id, as it didn't even begin 

[Maayan Harel]

Hi, 

I also tried to use the data and follow the instruction in the codelabhttps://codelabs.developers.google.com/codelabs/automl-forecasting-with-vertex-ai#0

and got the same error when I tried to initiate training - 

"Unable to start training due to the following error: Internal error encountered."

Thanks,

Maayan

[Joanne Qi]

Hi Maayan, what region are you running your jobs in? 

[Maayan Harel]

Hi,

Initially I tried in Europe regions and got the failures above. Yesterday I played with it and managed to run an example in a US region. Could it be that the service is available only on US regions or is this a bug?

Thanks,

Maayan

[Joanne Qi]

Yes, unfortunately there's an issue with regions outside of us-central1 and europe-west4 currently. The fix for that is being rolled out in the next few days, do you mind using us-central1 or europe-west4 in the meantime?

[Maayan Harel]

Hi,

Yes, I can use these regions. However, I'm getting now a different issue. I manage to train the model, but then when I attempt to do batch prediction, and try to load the source csv I get the message - "Source path must be in the same region as your model (us-central1)." Yet, I'm loading a dataset that was loaded to us-central1. Any suggestions on how to proceed? 

[Maayan Harel]

Hi,

Another issue that I have came across was when I tried to make batch predictions on multiple files (did it through the python interface with model.batch_predict).

To the best of my understanding, As I'm using a forecasting model for each forecast time I need to make a file that includes the labeled target and features for the context "window" and nans for the labels along the features for the forecasting range. For example, at forecast time 2020-01-03, if the context window is 2 hours and the forecast range is 2 hours:

target,  id,   x,  Time

3,          0,    5   2020-01-01

2,          0,    7   2020-01-02

Nan,    0,    7    2020-01-03

Nan,    0,    8.   2020-01-04

(In this example: x is a covariate, id is the timeseries identifier)

I mange to do successfully a batch prediction on a single file like this.

However, when I try to send multiple files (via a list in gcs_source argument in model.batch_predict) I get the validation error:

'There are rows with non-empty target values after this row. The time series has been excluded from predictions.'

No predictions are made.

I would appreciate if you could explain how batch prediction should be done in a timeseries forecasting model for multiple times and multiple forecast ranges.

Thanks,

Maayan

[Chenyu Zhao]

Hi Maayan,

1. Is it possible for you to share the file(s) you're sending to batch prediction?

2. Can you try combining the files into a single file to see if that works?

-Chenyu

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-automl-tables-discuss/ca0cb7ca-666c-46d0-bb74-c16e6e295a62n%40googlegroups.com.