I am running an
aggregate query with an "out" stage using Mongo java driver which
does not come back and keeps on running, however the
collection name given in the "out" stage
is created may be in 1 min. I don’t see any errors in the logs and it suggests
that the query is completed successfully.
I see the same kind of behavior in one of the Mongo client Robo Mongo as well.
Â
Looks to me like a strange behavior, any inputs on this would be much appreciated.
List<Bson> aggPipeline = Arrays.asList(
 Aggregates.unwind("$XRef"),
 Aggregates.lookup("product", "XRef.pkey", "rkey", "products"),
 Aggregates.group(new Document("mid", "$mid").append("Customer", "$Customer"), Accumulators.addToSet("XRef", "$XRef"), Accumulators.addToSet("products", "$products")),
 Aggregates.unwind("$products", new UnwindOptions().preserveNullAndEmptyArrays(true)),
 Aggregates.unwind("$products", new UnwindOptions().preserveNullAndEmptyArrays(true)),
 Aggregates.lookup("transaction", "products.pkey", "rkey", "products.transactions"),
 Aggregates.lookup("service-request", "products.pkey", "rkey", "products.srs"),
 Aggregates.group(new Document("mid", "$_id.mid").append("Customer", "$_id.Customer").append("XRef", "$XRef"), Accumulators.addToSet("products", "$products")),
 Aggregates.project(new Document("_id", "$_id.mid").append("mid", "$_id.mid").append("Customer", "$_id.Customer").append("XRef", "$_id.XRef").append("products", "$products")),
 Aggregates.out("customer_transactions_insert")
 );
customerInsertStgCollection.aggregate(aggPipeline).allowDiskUse(aggregationDiskUse).toCollection();
{
  "mid" : "1",
  "Customer" : {
    "source_id" : "CRM",
    "customer_id" : 2,
    "first_name" : "ANASTASIA",
    "middle_name" : "",
    "last_name" : "Deleon",
    "dob" : ISODate("1826-01-01T00:00:00.000Z"),
    "gender" : "Male",
    "address" : [
      {
        "source_id" : "CRM",
        "address_id" : 2,
        "address_type" : "Office",
        "address1" : "27652 East Armenia Ln.",
        "address2" : "dMtzrNFqjL",
        "address3" : "ZcGoPTNZrE",
        "city" : "St. Petersburg",
        "state" : "MT",
        "country" : "US",
        "zipcode" : 5136
      }
    ]
  },
  "XRef" : [
    {
      "SSID" : 1,
      "ID" : "2",
      "pkey" : "1-2"
    }
  ]
}
{
  "product_id" : "3071",
  "issue_date" : ISODate("2016-03-06T00:00:00.000Z"),
  "valid_till_date" : Date(253370764800000),
  "account_status" : "open",
  "customer_id" : "2",
  "SSID" : 1,
  "pkey" : "1-3071",
  "rkey" : "1-2"
}
{
  "transaction_id" : "3",
  "customer_id" : "62",
  "product_id" : "670",
  "transaction_amount" : 2975.0,
  "transaction_date" : ISODate("2016-09-02T00:00:00.000Z"),
  "SSID" : 1,
  "pkey" : "1-3",
  "rkey" : "1-670"
}
{
  "sr_id" : "2602",
  "customer_id" : "2",
  "product_id" : "3071",
  "sr_open_date" : ISODate("2016-04-06T00:00:00.000Z"),
  "sr_close_date " : ISODate("2016-04-06T00:00:00.000Z"),
  "sr_type" : "Complaint",
  "sr_cust_feedback" : "the complaint was addressed in timely manner to my satisfaction",
  "SSID" : 2,
  "pkey" : "2-2602",
  "rkey" : "1-3071"
}
db.getCollection('customer').aggregate([
{
  $unwind: "$XRef"
},
{
  $lookup: {
    from: "product",
    localField: "XRef.pkey",
    foreignField: "rkey",
    as: "products"
  }
},
{
  $group: {
    _id: {mid: "$mid", Customer: "$Customer"},
    XRef: {$addToSet: "$XRef"},
    products:{$addToSet: "$products"}
  }
},
{
  $unwind: {
    "path": "$products",
    "preserveNullAndEmptyArrays": true
  }
},
{
  $unwind: {
    "path": "$products",
    "preserveNullAndEmptyArrays": true
  }
},
{
  $lookup: {
    from: "transaction",
    localField: "products.pkey",
    foreignField: "rkey",
    as: "products.transactions"
  }
},
{
  $lookup: {
    from: "service-request",
    localField: "products.pkey",
    foreignField: "rkey",
    as: "products.srs"
  }
},
{
  $project: {
    mid: "$_id.mid",
    Customer: "$_id.Customer",
    XRef: "$XRef",
    products: "$products",
    _id: 0
  }
},
{
  $out: "cust_tran"
}
],
{
  allowDiskUse : true
}
)
Any update on this please.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/34acc457-0437-49fb-a90d-5575a9632c92%40googlegroups.com.
Hi Asya,
Query runs perfectly fine in Mongo shell and the control comes back whereas the same query has issues when run using Java driver.
Ravi
>> email to mongodb-user+unsubscribe@googlegroups.com.
>> To post to this group, send email to mongod...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/mongodb-user.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/mongodb-user/34acc457-0437-49fb-a90d-5575a9632c92%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user"
> group.
>
> For other MongoDB technical support options, see:
> https://docs.mongodb.com/manual/support/
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mongodb-user+unsubscribe@googlegroups.com.
> To post to this group, send email to mongod...@googlegroups.com.
> Visit this group at https://groups.google.com/group/mongodb-user.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mongodb-user/CA%2BDEm-e%2B2t%3DaWwxo-4Cgbs8D2UUa37RUbYxWCFWzMqdwB%2BJc2A%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.
--
Asya Kamsky
Lead Product Manager
MongoDB
Download MongoDB - mongodb.org/downloads
We're Hiring! - https://www.mongodb.com/careers
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJDdc97xVtcLEi1adgNF6TdBxm_129ZGR3ziEkd4PyYTOQ%40mail.gmail.com.