If I understood it correctly, this is just for select queries right ?
(no transactional querying). Will there be additional operations like
update,delete etc be available in future (if not already present) ?
> If I understood it correctly, this is just for select queries right ?
> (no transactional querying). Will there be additional operations like
> update,delete etc be available in future (if not already present) ?
I guess we need to be careful about how we answer questions... So far at
Google I've always just answered assuming everyone is a Googler. Now we'll
have to check to TO line and FROM line to see how we need to answer a
question.
Good learning for all!
--Michael Sheldon
On Thu, May 20, 2010 at 1:26 AM, Amit Agarwal <amit.agar...@gmail.com>wrote:
> BigQuery is just for select queries. We currently dont have any plans for
> additional operations like update, delete etc. at a transaction level.
> thanks
> Amit
> On Thu, May 20, 2010 at 1:24 AM, Mahesh Velaga <maheshraj...@gmail.com>wrote:
>> Hi,
>> If I understood it correctly, this is just for select queries right ?
>> (no transactional querying). Will there be additional operations like
>> update,delete etc be available in future (if not already present) ?
In our case we have a data set which increases at the rate of about 52
million records per year, and 150,000 per day (updated twice per
day). If I uploaded the initial data set is it possible to append
additional data? Update and delete would be awesome though (I can
think of other uses).
Additionally is it possible to pivots on data using this query
language? I don't have an account (sad face) and I couldn't work it
out from the docs.
On May 20, 6:22 pm, Michael Sheldon <mshel...@google.com> wrote:
> I guess we need to be careful about how we answer questions... So far at
> Google I've always just answered assuming everyone is a Googler. Now we'll
> have to check to TO line and FROM line to see how we need to answer a
> question.
> Good learning for all!
> --Michael Sheldon
> On Thu, May 20, 2010 at 1:26 AM, Amit Agarwal <amit.agar...@gmail.com>wrote:
> > Hi Mahesh,
> > BigQuery is just for select queries. We currently dont have any plans for
> > additional operations like update, delete etc. at a transaction level.
> > thanks
> > Amit
> > On Thu, May 20, 2010 at 1:24 AM, Mahesh Velaga <maheshraj...@gmail.com>wrote:
> >> Hi,
> >> If I understood it correctly, this is just for select queries right ?
> >> (no transactional querying). Will there be additional operations like
> >> update,delete etc be available in future (if not already present) ?
Thank you for the interest in BigQuery. Let me try and answer your questions:
Yes, you can append additional records multiple times a day to your BigQuery Tables. (We intend to have a "continuous import" feature to make this easier by letting you configure your BigQuery Table to automatically import all data you push into your Google Storage for Developers bucket. However that feature is a bit further out.)
Our query language does not support Pivots. Thanks for asking: we'll take that as a feature request.
Regarding accounts:
- You can start by making sure you have a Google Storage for Developers account. You can find the sign up link for this service at http://code.google.com/apis/storage/. - You can find the BigQuery sign up link near the bottom of http://code.google.com/apis/bigquery/. We had more interest in our service than we anticipated, so it will take a while before we can provision accounts for everyone interested. We will get back to everyone that signs up. Thanks for your patience!
On Sat, May 22, 2010 at 4:45 PM, Alex Moore <zycl...@gmail.com> wrote: > In our case we have a data set which increases at the rate of about 52 > million records per year, and 150,000 per day (updated twice per > day). If I uploaded the initial data set is it possible to append > additional data? Update and delete would be awesome though (I can > think of other uses).
> Additionally is it possible to pivots on data using this query > language? I don't have an account (sad face) and I couldn't work it > out from the docs.
> > On Thu, May 20, 2010 at 1:26 AM, Amit Agarwal <amit.agar...@gmail.com > >wrote:
> > > Hi Mahesh,
> > > BigQuery is just for select queries. We currently dont have any plans > for > > > additional operations like update, delete etc. at a transaction level.
> > > thanks > > > Amit
> > > On Thu, May 20, 2010 at 1:24 AM, Mahesh Velaga <maheshraj...@gmail.com > >wrote:
> > >> Hi,
> > >> If I understood it correctly, this is just for select queries right ? > > >> (no transactional querying). Will there be additional operations like > > >> update,delete etc be available in future (if not already present) ?
It was so amazing to see that the queries for 60B records are done in a few seconds. What kind of approach do you use inside it?
a. on-demand massively parallel query with thousands of nodes b. pre-aggregation/computation (like building B-Trees for all the columns) at the import time c. any other unknown technique
And how long did it take to import the 60B records?
Thanks,
Kaz
On 5月26日, 午前3:52, Michael Sheldon <mshel...@google.com> wrote:
> Thank you for the interest in BigQuery. Let me try and answer your > questions:
> Yes, you can append additional records multiple times a day to your BigQuery > Tables. (We intend to have a "continuous import" feature to make this easier > by letting you configure your BigQuery Table to automatically import all > data you push into your Google Storage for Developers bucket. However that > feature is a bit further out.)
> Our query language does not support Pivots. Thanks for asking: we'll take > that as a feature request.
> Regarding accounts:
> - You can start by making sure you have a Google Storage for Developers > account. You can find the sign up link for this service at > http://code.google.com/apis/storage/. > - You can find the BigQuery sign up link near the bottom of > http://code.google.com/apis/bigquery/. We had more interest in our > service than we anticipated, so it will take a while before we can provision > accounts for everyone interested. We will get back to everyone that signs > up. Thanks for your patience!
> Cheers,
> --Michael Sheldon
> On Sat, May 22, 2010 at 4:45 PM, Alex Moore <zycl...@gmail.com> wrote: > > In our case we have a data set which increases at the rate of about 52 > > million records per year, and 150,000 per day (updated twice per > > day). If I uploaded the initial data set is it possible to append > > additional data? Update and delete would be awesome though (I can > > think of other uses).
> > Additionally is it possible to pivots on data using this query > > language? I don't have an account (sad face) and I couldn't work it > > out from the docs.
> > > On Thu, May 20, 2010 at 1:26 AM, Amit Agarwal <amit.agar...@gmail.com > > >wrote:
> > > > Hi Mahesh,
> > > > BigQuery is just for select queries. We currently dont have any plans > > for > > > > additional operations like update, delete etc. at a transaction level.
> > > > thanks > > > > Amit
> > > > On Thu, May 20, 2010 at 1:24 AM, Mahesh Velaga <maheshraj...@gmail.com > > >wrote:
> > > >> Hi,
> > > >> If I understood it correctly, this is just for select queries right ? > > > >> (no transactional querying). Will there be additional operations like > > > >> update,delete etc be available in future (if not already present) ?
Yes, 60B is what really drew my attention to it :)
Will BigQuery be available for "almost" real time reporting? How long will it take (i.e. is it planned to take) for the data to be "query-able" in BigQuery after the data has been uploaded to google storage?
Any ideas on what the pricing model is going to be like? Per query, per amount of data?
On May 27, 12:46 am, kazunori_279 <kazunori...@gmail.com> wrote:
> It was so amazing to see that the queries for 60B records are done in > a few seconds. > What kind of approach do you use inside it?
> a. on-demand massively parallel query with thousands of nodes > b. pre-aggregation/computation (like building B-Trees for all the > columns) at the import time > c. any other unknown technique
> And how long did it take to import the 60B records?
> Thanks,
> Kaz
> On 5月26日, 午前3:52, Michael Sheldon <mshel...@google.com> wrote:
> > Hello Alex,
> > Thank you for the interest in BigQuery. Let me try and answer your > > questions:
> > Yes, you can append additional records multiple times a day to your BigQuery > > Tables. (We intend to have a "continuous import" feature to make this easier > > by letting you configure your BigQuery Table to automatically import all > > data you push into your Google Storage for Developers bucket. However that > > feature is a bit further out.)
> > Our query language does not support Pivots. Thanks for asking: we'll take > > that as a feature request.
> > Regarding accounts:
> > - You can start by making sure you have a Google Storage for Developers > > account. You can find the sign up link for this service at > > http://code.google.com/apis/storage/. > > - You can find the BigQuery sign up link near the bottom of > > http://code.google.com/apis/bigquery/. We had more interest in our > > service than we anticipated, so it will take a while before we can provision > > accounts for everyone interested. We will get back to everyone that signs > > up. Thanks for your patience!
> > Cheers,
> > --Michael Sheldon
> > On Sat, May 22, 2010 at 4:45 PM, Alex Moore <zycl...@gmail.com> wrote: > > > In our case we have a data set which increases at the rate of about 52 > > > million records per year, and 150,000 per day (updated twice per > > > day). If I uploaded the initial data set is it possible to append > > > additional data? Update and delete would be awesome though (I can > > > think of other uses).
> > > Additionally is it possible to pivots on data using this query > > > language? I don't have an account (sad face) and I couldn't work it > > > out from the docs.
> > > > On Thu, May 20, 2010 at 1:26 AM, Amit Agarwal <amit.agar...@gmail.com > > > >wrote:
> > > > > Hi Mahesh,
> > > > > BigQuery is just for select queries. We currently dont have any plans > > > for > > > > > additional operations like update, delete etc. at a transaction level.
> > > > > thanks > > > > > Amit
> > > > > On Thu, May 20, 2010 at 1:24 AM, Mahesh Velaga <maheshraj...@gmail.com > > > >wrote:
> > > > >> Hi,
> > > > >> If I understood it correctly, this is just for select queries right ? > > > > >> (no transactional querying). Will there be additional operations like > > > > >> update,delete etc be available in future (if not already present) ?
We don't comment on the backend details of BigQuery. What I can say is that we are running BigQuery on thousands of machines.
The 60 B records were imported as part of a sample dataset. We don't have numbers to share at this point regarding the time taken to import the dataset.
On Wed, May 26, 2010 at 8:46 AM, kazunori_279 <kazunori...@gmail.com> wrote: > Hi Micheal,
> It was so amazing to see that the queries for 60B records are done in > a few seconds. > What kind of approach do you use inside it?
> a. on-demand massively parallel query with thousands of nodes > b. pre-aggregation/computation (like building B-Trees for all the > columns) at the import time > c. any other unknown technique
> And how long did it take to import the 60B records?
> Thanks,
> Kaz
> On 5月26日, 午前3:52, Michael Sheldon <mshel...@google.com> wrote: > > Hello Alex,
> > Thank you for the interest in BigQuery. Let me try and answer your > > questions:
> > Yes, you can append additional records multiple times a day to your > BigQuery > > Tables. (We intend to have a "continuous import" feature to make this > easier > > by letting you configure your BigQuery Table to automatically import all > > data you push into your Google Storage for Developers bucket. However > that > > feature is a bit further out.)
> > Our query language does not support Pivots. Thanks for asking: we'll take > > that as a feature request.
> > Regarding accounts:
> > - You can start by making sure you have a Google Storage for > Developers > > account. You can find the sign up link for this service at > > http://code.google.com/apis/storage/. > > - You can find the BigQuery sign up link near the bottom of > > http://code.google.com/apis/bigquery/. We had more interest in our > > service than we anticipated, so it will take a while before we can > provision > > accounts for everyone interested. We will get back to everyone that > signs > > up. Thanks for your patience!
> > Cheers,
> > --Michael Sheldon
> > On Sat, May 22, 2010 at 4:45 PM, Alex Moore <zycl...@gmail.com> wrote: > > > In our case we have a data set which increases at the rate of about 52 > > > million records per year, and 150,000 per day (updated twice per > > > day). If I uploaded the initial data set is it possible to append > > > additional data? Update and delete would be awesome though (I can > > > think of other uses).
> > > Additionally is it possible to pivots on data using this query > > > language? I don't have an account (sad face) and I couldn't work it > > > out from the docs.
> > > > On Thu, May 20, 2010 at 1:26 AM, Amit Agarwal < > amit.agar...@gmail.com > > > >wrote:
> > > > > Hi Mahesh,
> > > > > BigQuery is just for select queries. We currently dont have any > plans > > > for > > > > > additional operations like update, delete etc. at a transaction > level.
> > > > > thanks > > > > > Amit
> > > > > On Thu, May 20, 2010 at 1:24 AM, Mahesh Velaga < > maheshraj...@gmail.com > > > >wrote:
> > > > >> Hi,
> > > > >> If I understood it correctly, this is just for select queries > right ? > > > > >> (no transactional querying). Will there be additional operations > like > > > > >> update,delete etc be available in future (if not already present) > ?
Our first priority is get the ability to import data into BigQuery.
On Wed, May 26, 2010 at 4:42 PM, Alex Moore <zycl...@gmail.com> wrote: > Yes, 60B is what really drew my attention to it :)
> Will BigQuery be available for "almost" real time reporting? > How long will it take (i.e. is it planned to take) for the data to be > "query-able" in BigQuery after the data has been uploaded to google > storage?
Real time has different meanings for different users and different use cases. We are working on making the data "query-able" as fast as possible once it has been uploaded to Google Storage for developers.
> Any ideas on what the pricing model is going to be like? Per query, > per amount of data?
Currently BigQuery is in preview and we don't have anything to share on pricing. During this preview period we want to understand and tune the system before we work on pricing.
> On May 27, 12:46 am, kazunori_279 <kazunori...@gmail.com> wrote: > > Hi Micheal,
> > It was so amazing to see that the queries for 60B records are done in > > a few seconds. > > What kind of approach do you use inside it?
> > a. on-demand massively parallel query with thousands of nodes > > b. pre-aggregation/computation (like building B-Trees for all the > > columns) at the import time > > c. any other unknown technique
> > And how long did it take to import the 60B records?
> > Thanks,
> > Kaz
> > On 5月26日, 午前3:52, Michael Sheldon <mshel...@google.com> wrote:
> > > Hello Alex,
> > > Thank you for the interest in BigQuery. Let me try and answer your > > > questions:
> > > Yes, you can append additional records multiple times a day to your > BigQuery > > > Tables. (We intend to have a "continuous import" feature to make this > easier > > > by letting you configure your BigQuery Table to automatically import > all > > > data you push into your Google Storage for Developers bucket. However > that > > > feature is a bit further out.)
> > > Our query language does not support Pivots. Thanks for asking: we'll > take > > > that as a feature request.
> > > Regarding accounts:
> > > - You can start by making sure you have a Google Storage for > Developers > > > account. You can find the sign up link for this service at > > > http://code.google.com/apis/storage/. > > > - You can find the BigQuery sign up link near the bottom of > > > http://code.google.com/apis/bigquery/. We had more interest in our > > > service than we anticipated, so it will take a while before we can > provision > > > accounts for everyone interested. We will get back to everyone that > signs > > > up. Thanks for your patience!
> > > Cheers,
> > > --Michael Sheldon
> > > On Sat, May 22, 2010 at 4:45 PM, Alex Moore <zycl...@gmail.com> wrote: > > > > In our case we have a data set which increases at the rate of about > 52 > > > > million records per year, and 150,000 per day (updated twice per > > > > day). If I uploaded the initial data set is it possible to append > > > > additional data? Update and delete would be awesome though (I can > > > > think of other uses).
> > > > Additionally is it possible to pivots on data using this query > > > > language? I don't have an account (sad face) and I couldn't work it > > > > out from the docs.
> > > > > On Thu, May 20, 2010 at 1:26 AM, Amit Agarwal < > amit.agar...@gmail.com > > > > >wrote:
> > > > > > Hi Mahesh,
> > > > > > BigQuery is just for select queries. We currently dont have any > plans > > > > for > > > > > > additional operations like update, delete etc. at a transaction > level.
> > > > > > thanks > > > > > > Amit
> > > > > > On Thu, May 20, 2010 at 1:24 AM, Mahesh Velaga < > maheshraj...@gmail.com > > > > >wrote:
> > > > > >> Hi,
> > > > > >> If I understood it correctly, this is just for select queries > right ? > > > > > >> (no transactional querying). Will there be additional operations > like > > > > > >> update,delete etc be available in future (if not already > present) ?
