Get asset metadata via API

Ross Bender

unread,

Jun 2, 2018, 6:58:31 PM6/2/18

to Nexus Users

Is there any means by which I can get asset metadata via the API? For example the last modified date and Maven classifier. I know the second is an input parameter for the search components API but I don't know of any way to retrieve the former.

I'm using the search API to retrieve the component I want to take action on, but it'd be nice if the asset/{id} API returned full detail about the asset, like the GUI displays.

Ross Bender

unread,

Jun 10, 2018, 2:38:46 PM6/10/18

to Nexus Users

Just curious if anyone had thoughts on this?

Joseph Stephens

unread,

Jun 11, 2018, 7:07:00 AM6/11/18

to Ross Bender, Nexus Users

Hi Ross,

I can't see a way currently to get the asset attribute information via the REST API (there may be one but I'm not aware of one). As an alternative you could take advantage of the scripting API https://help.sonatype.com/repomanager3/rest-and-integration-api/script-api/managing-and-running-scripts

Here is an example script, that lists ALL assets for a given repository, which you can adapt to get a single asset or even a single field (such as classifier) from an asset - https://gist.github.com/kellyrob99/2d1483828c5de0e41732327ded3ab224 .

StorageTx (tx) has a method findAssetWithProperty instead of findAssets i.e.

findAssetWithProperty('name', 'ant/ant/1.7.0/ant-1.7.0.jar')

You can find the fields you need in asset.attributes()

Hope this helps,

Joe

On Sun, Jun 10, 2018 at 3:38 PM, Ross Bender <bende...@gmail.com> wrote:

Just curious if anyone had thoughts on this?

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/fbe84179-a2cc-44f3-9794-42489d7e522b%40glists.sonatype.com.

For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

--

Joseph Stephens

Wes Wannemacher

unread,

Jun 11, 2018, 9:52:51 AM6/11/18

to Ross Bender, Nexus Users

Don't forget to close the transaction on that script:

https://support.sonatype.com/hc/en-us/articles/115015847987-Groovy-Close-those-transactions-or-else-

-Wes

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/CABs6fL90iCUvRB6EPTuQsd2XwBgYR1bk0kOR-cr%3DaTgzsci9GQ%40mail.gmail.com.

Ross Bender

unread,

Jun 21, 2018, 2:02:41 PM6/21/18

to Nexus Users

I've been playing around more with this problem and it seems like my script is not returning data when searching a group type repository. I can get data back if searching directly against a hosted repo.

Is this by design or is it a bug?

Ryan Smith

unread,

Jul 11, 2018, 11:06:25 AM7/11/18

to Nexus Users

I'm quite curious as to how far you've progressed towards a solution, Ross. I'm trying currently to access all of my assets and get a list of the Uploaders for all of them and am stalling out. Did you end up finding a/creating your own solution?

Ross Bender

unread,

Jul 11, 2018, 2:35:40 PM7/11/18

to Nexus Users

Yeah, I ended up creating my own solution to retrieve assets with the data I needed. I built off some of the examples for retrieving assets and expanded it so that it returned some things like when the asset was created. Since we're using Maven, I also pull only the latest version for snapshot builds. Here's a sample input/output:

Request:

{
    "repoName": "frontend-snapshots",
    "groupId": "org.company",
    "artifactId": "my-app",
    "startDate": "2018-06-20"
}

Response:

[{
        "groupId": "org.company",
        "artifactId": "my-app",
        "version": "v1.0.9",
        "updated": "2018-07-06T12:47:45.655-05:00",
        "files": [{
                "classifier": null,
                "fileExtension": "jar",
                "path": "/repository/frontend-releases/org/company/my-app/v1.0.9/my-app-v1.0.9.jar"
            }, {
                "classifier": null,
                "fileExtension": "jar.sha1",
                "path": "/repository/frontend-releases/org/company/my-app/v1.0.9/my-app-v1.0.9.jar.sha1"
            }, {
                "classifier": null,
                "fileExtension": "jar.md5",
                "path": "/repository/frontend-releases/org/company/my-app/v1.0.9/my-app-v1.0.9.jar.md5"
            }, {
                "classifier": null,
                "fileExtension": "pom.sha1",
                "path": "/repository/frontend-releases/org/company/my-app/v1.0.9/my-app-v1.0.9.pom.sha1"
            }, {
                "classifier": null,
                "fileExtension": "pom",
                "path": "/repository/frontend-releases/org/company/my-app/v1.0.9/my-app-v1.0.9.pom"
            }, {
                "classifier": null,
                "fileExtension": "pom.md5",
                "path": "/repository/frontend-releases/org/company/my-app/v1.0.9/my-app-v1.0.9.pom.md5"
            }
        ]
    }
]

In regard to the issue about trying to load assets from a "group" type repository, my workaround was to have my application iterate the individual repos for a group and call the script for each one.

If you're looking for the uploader for an asset, I think that's a property you can get directly from the Nexus Asset object using the createdBy() method. As you'll see in my script below, there were some properties I was able to get directly from the Asset object, and others I had to get from its "attribute" map.

Hope that helps!

Full script below:

import org.joda.time.DateTime
import org.sonatype.nexus.common.collect.DetachingMap
import org.sonatype.nexus.repository.storage.Asset
import org.sonatype.nexus.repository.storage.Query
import org.sonatype.nexus.repository.storage.StorageFacet
import org.sonatype.nexus.repository.storage.StorageTx

import groovy.json.JsonBuilder
import groovy.json.JsonSlurper

class MavenArtifact {
    String group
    String id
    String specificVersion
    String baseVersion
    DateTime lastUpdated
    List<MavenArtifactAsset> assets = new ArrayList<MavenArtifactAsset>()
    
    String specificGav() {
        "${group}:${id}:${specificVersion}"
    }
    String baseGav() {
        "${group}:${id}:${baseVersion}"
    }
}
class MavenArtifactAsset {
    String path
    String fileExtension
    String classifier
}

def request = new JsonSlurper().parseText(args)
assert request.repoName: 'repoName parameter is required'
assert request.groupId: 'groupId parameter is required'
assert request.artifactId: 'artifactId parameter is required'
assert request.startDate: 'startDate parameter is required, format: yyyy-mm-dd'

log.info("Asset query: repoName=${request.repoName}, groupId=${request.groupId}, artifactId=${request.artifactId}, startDate=${request.startDate}")

def repo = repository.repositoryManager.get(request.repoName)
StorageFacet storageFacet = repo.facet(StorageFacet)
StorageTx tx = storageFacet.txSupplier().get()


List<Asset> allAssets = new ArrayList<Asset>()
try {
    tx.begin()
    
    // the following URL was extremely helpful for query building: https://github.com/sonatype/nexus-public/blob/a6a8abcdcba3fd1947884f05f913d0da6939261c/plugins/nexus-repository-maven/src/main/java/org/sonatype/nexus/repository/maven/internal/MavenIndexPublisher.java#L100
    Iterable<Asset> results = tx.findAssets(
        Query.builder()
            .where('component.group').eq(request.groupId)
            .and('component.name').eq(request.artifactId)
            .and('blob_created > ').param(request.startDate)
        .build(), [repo])
    
    results.each {
        allAssets.add(it)
    }
} finally {
    tx.close()
}

Map<String, MavenArtifact> artifactsBySpecificVersion = new HashMap<String, MavenArtifact>()
log.info("${allAssets.size()} total assets found")

allAssets.each {
    def mavenProps = it.attributes().get("maven2")
    if (mavenProps) {
        log.debug("${it.name()}")
        DetachingMap mavenPropMap = mavenProps.asType(DetachingMap)
        /*
        mavenPropMap.each { mavenProp ->
            log.info("\t${mavenProp.getKey()}: ${mavenProp.getValue()} (${mavenProp.getValue().getClass().getSimpleName()})")
        }
        */
        
        MavenArtifact ma = new MavenArtifact()
        ma.group = mavenPropMap.get("groupId")
        ma.id = mavenPropMap.get("artifactId")
        ma.specificVersion = mavenPropMap.get("version")
        ma.baseVersion = mavenPropMap.get("baseVersion")
        ma.lastUpdated = it.blobCreated()
        if (!artifactsBySpecificVersion.containsKey(ma.specificGav())) {
            artifactsBySpecificVersion.put(ma.specificGav(), ma)
        }
        List<MavenArtifactAsset> artifactAssets = artifactsBySpecificVersion.get(ma.specificGav()).assets
        MavenArtifactAsset maa = new MavenArtifactAsset()
        maa.path = "/repository/${repo.name}/${it.name()}"
        maa.fileExtension = mavenPropMap.get("extension")
        maa.classifier = mavenPropMap.get("classifier")
        artifactAssets.add(maa)
    } else {
        log.warn("no maven props found for ${it.name()}")
    }
}

List<MavenArtifact> specificVersionArtifacts = new ArrayList<MavenArtifact>(artifactsBySpecificVersion.values())
log.info("${specificVersionArtifacts.size()} total artifacts found")

specificVersionArtifacts.each {
    log.debug("\t${it.specificGav()} has ${it.assets.size()} assets")
}

specificVersionArtifacts.sort { a, b -> b.lastUpdated <=> a.lastUpdated } // sort by last updated descending

log.debug("after sorting:")
specificVersionArtifacts.each {
    log.debug("\t${it.specificGav()} has ${it.assets.size()} assets")
}

log.debug("removing snapshot artifacts")
Map<String, MavenArtifact> artifactsByBaseVersion = new HashMap<String, MavenArtifact>()
specificVersionArtifacts.each {
    // since list is ordered, only keep first (AKA most recent) artifact for same base version. this removes any old 'snapshot' artifacts
    if (!artifactsByBaseVersion.containsKey(it.baseGav())) {
        artifactsByBaseVersion.put(it.baseGav(), it)
    }
}

List<MavenArtifact> baseVersionArtifacts = new ArrayList<MavenArtifact>(artifactsByBaseVersion.values())
baseVersionArtifacts.sort { a, b -> b.lastUpdated <=> a.lastUpdated } // sort by last updated descending
log.info("${baseVersionArtifacts.size()} unique artifacts found")

baseVersionArtifacts.each {
    log.debug("\t${it.specificGav()} has ${it.assets.size()} assets")
}

JsonBuilder jsonBuilder = new JsonBuilder()

jsonBuilder(
    baseVersionArtifacts.collect {
        [
            groupId: it.group,
            artifactId: it.id,
            version: it.baseVersion,
            updated: it.lastUpdated.toString(),
            files: it.assets
        ]
    }
)

return jsonBuilder.toString()

Message has been deleted

Ross Bender

unread,

Jul 11, 2018, 2:40:17 PM7/11/18

to Nexus Users

Not sure if the whole script came through in the post. Attaching it as a file.

get-maven-artifacts.groovy

Ryan Smith

unread,

Jul 11, 2018, 3:16:30 PM7/11/18

to Nexus Users

This EXTREMELY helpful Ross. Thank you so much! Nothing was really clicking for me but seeing it all laid out is helping a ton.

diptim...@appdynamics.com

unread,

Oct 25, 2018, 5:46:15 AM10/25/18

to Nexus Users

Hi,

Can you share the file with me. It will be really helpful.

Thanks

Diptiman

diptim...@appdynamics.com

unread,

Oct 25, 2018, 6:27:46 AM10/25/18

to Nexus Users

I am not understanding how do you initialise repository.

def repo = repository.repositoryManager.get(request.repoName).

It will be great if you can give some insight for this.

Thanks

Diptiman Adak

Reply all

Reply to author

Forward