Fetch checksum for single file in Java CLI

[Stephen McConnachie]

Hi everyone,

I am trying to find a method that will let me use the CLI to fetch a checksum from Black Pearl for a single object. 

I tried head_object but the etag or whatever the checksum is labelled, is not among the keys returned.

I also tried get_detailed_objects with a filter-params contains and the object name -as below - but get repeated iterations of a 200 response with no information.

Is there another method to fetch object metadata including checksum?

Cheers,

Stephen

~$ /mnt/isilon/ds3_java_cli -e IP_REDACTED -a KEYS_REDACTED -c get_detailed_objects -b bucket --filter-params contains:0x060a2b340101010201010f1213f8af85aa12c503058505805448645106EF7E9C --http --verbose

2021-11-01 16:49:11 +++ Version: 5.1.2

2021-11-01 16:49:11 +++ Build Date: Fri Oct 16 15:22:21 MDT 2020

2021-11-01 16:49:11 +++ Command line args: -e, IP_REDACTED, -a, KEY_REDACTED, -c, get_detailed_objects, -b, bucket, --filter-params, contains:0x060a2b340101010201010f1213f8af85aa12c503058505805448645106EF7E9C, --http, --verbose

2021-11-01 16:49:11 +++ Console log level: INFO

2021-11-01 16:49:11 +++ Log file log level: OFF

2021-11-01 16:49:11 +++ Java Version: {1.8.0_222}

Java Vendor: {Private Build}

JVM Version: {25.222-b10}

JVM Name: {OpenJDK 64-Bit Server VM}

OS: {Linux}

OS Arch: {amd64}

OS Version: {3.19.0-80-generic}

2021-11-01 16:49:11 +++ Making connection details for endpoint [IP_REDACTED] using this authorization id [KEY_REDACTED]

2021-11-01 16:49:11 +++ Sending request: GET http://IP_REDACTED:80 /_rest_/system_information

2021-11-01 16:49:11 +++ Server responded with 200 for request #4069509

2021-11-01 16:49:12 +++ Sending request: GET http://IP_REDACTED:80 /_rest_/object

2021-11-01 16:49:14 +++ Server responded with 200 for request #4069510

2021-11-01 16:49:14 +++ Sending request: GET http://IP_REDACTED:80 /_rest_/object

2021-11-01 16:49:16 +++ Server responded with 200 for request #4069512

etc etc etc etc

[George Trossell]

Hi Stephen, 

We have looked into this for you and it looks like you need to change the format output to JSON to get the etag, which is the MD5 you are looking for.

Please see the example command and output below

ds3_java_cli --http -c get_detailed_objects -b seans --filter-params contains:raccoon.jpg --output-format json --verbose
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/Users/seans/Documents/Programs/ds3_java_cli-5.1.2/lib/guice-4.2.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2022-02-03 11:29:39 +++ Version: 5.1.2
2022-02-03 11:29:39 +++ Build Date: Fri Oct 16 15:23:48 MDT 2020
2022-02-03 11:29:39 +++ Command line args: --http, -c, get_detailed_objects, -b, seans, --filter-params, contains:raccoon.jpg, --output-format, json, --verbose
2022-02-03 11:29:39 +++ Console log level: INFO
2022-02-03 11:29:39 +++ Log file log level: OFF
2022-02-03 11:29:39 +++ Java Version: {14}
Java Vendor: {Oracle Corporation}
JVM Version: {14+36-1461}

JVM Name: {OpenJDK 64-Bit Server VM}

OS: {Mac OS X}
OS Arch: {x86_64}
OS Version: {10.16}

2022-02-03 11:29:39 +++ Making connection details for endpoint [10.85.42.41] using this authorization id [c2VhbnM=]
2022-02-03 11:29:39 +++ Sending request: GET http://10.85.42.41:80 /_rest_/system_information
2022-02-03 11:29:40 +++ Server responded with 200 for request #173899
2022-02-03 11:29:40 +++ Sending request: GET http://10.85.42.41:80 /_rest_/object
2022-02-03 11:29:41 +++ Server responded with 200 for request #173900
{
  "Meta" : {
    "Date" : "2022-02-03T16:29:40.312Z"
  },
  "Data" : {
    "result" : [ {
      "etag" : "bd97d29def6948c27d9bd9835f6185b6",  <- This is the MD5 of the object we are looking for in the command
      "Blobs" : {
        "Object" : [ {
          "bucket" : "seans",
          "id" : "25e6809b-65c5-407e-9265-02e5395c8e7a",
          "inCache" : false,
          "latest" : true,
          "length" : 266885,
          "name" : "raccoon.jpg",
          "offset" : 0,
          "versionId" : "6d2ea486-fb32-4599-8832-a78c2280c8c6",
          "PhysicalPlacement" : {
            "AzureTargets" : null,
            "Ds3Targets" : null,
            "Pools" : null,
            "S3Targets" : null,
            "Tapes" : [ {
              "AssignedToStorageDomain" : true,
              "AvailableRawCapacity" : 0,
              "BarCode" : "430115L5",
              "BucketId" : null,
              "DescriptionForIdentification" : "b94aa6e6-f434-4adc-9ba6-66d319670ed6:BlackPearl@f0110c7e-4ad5-488b-9818-571317dff9dc",
              "EjectDate" : null,
              "EjectLabel" : null,
              "EjectLocation" : null,
              "EjectPending" : null,
              "FullOfData" : true,
              "Id" : "f0110c7e-4ad5-488b-9818-571317dff9dc",
              "LastAccessed" : "2022-01-19T22:26:28.579Z",
              "LastCheckpoint" : "b94aa6e6-f434-4adc-9ba6-66d319670ed6:1484",
              "LastModified" : "2021-10-26T15:13:25.085Z",
              "LastVerified" : null,
              "PartiallyVerifiedEndOfTape" : null,
              "PartitionId" : "b5999653-1d9c-4ab2-bf40-68d7088768d4",
              "PreviousState" : null,
              "SerialNumber" : "HP-G140706024",
              "State" : "NORMAL",
              "StorageDomainMemberId" : "52f79891-33ef-478b-9afb-d0a0ce549c7e",
              "TakeOwnershipPending" : false,
              "TotalRawCapacity" : 1425000103936,
              "Type" : "LTO5",
              "VerifyPending" : null,
              "WriteProtected" : false
            } ]
          }
        } ]
      },
      "BlobsBeingPersisted" : 0,
      "BlobsDegraded" : 0,
      "BlobsInCache" : 0,
      "BlobsTotal" : 1,
      "BucketId" : "a60f002f-2993-44b9-9c67-d239832e6ece",
      "CreationDate" : "2021-10-12T19:14:45.100Z",
      "ETag" : "bd97d29def6948c27d9bd9835f6185b6",  <- This is the MD5 of the object we are looking for in the command
      "Id" : "6d2ea486-fb32-4599-8832-a78c2280c8c6",
      "Latest" : true,
      "Name" : "raccoon.jpg",
      "Owner" : "Administrator",
      "Size" : 266885,
      "Type" : "DATA"
    } ]
  },
  "Status" : "OK"
}

Kind regards

George