[Soot-list] Missing callgraph edges with Spark

114 views
Skip to first unread message

Faridah Akinotcho

unread,
May 8, 2022, 6:10:45 PM5/8/22
to Soot-list
Hi,

I am using Soot (org.soot.oss 4.2.1) to perform an interprocedural analysis on Android methods (in the wjtp phase). To this end, I am extracting reachable methods from Android lifecycles using Scene.v().getCallgraph().edgesOutOf(method). However, I noticed that some virtual and special edges seem to always be missing from the returned set. For e.g, given:

class B extends A{
    void onCreate(...){
        setContentView(C0027R.layout.merge);
        super.onCreate(bundle);
        setNextActivity(C.class);
    }
}

class A{
    public void setNextActivity(){
        ...
    }
}

When computing the reachable methods for B.onCreate(), neither setContentView, super.onCreate or setNextActivity are included as targets for the outgoing edges (only clinit is returned). However, when using CHA instead of Spark, these are part of the returned set (along other overapproximations), so I am assuming there might be some issue when performing the points-to analysis. Here are the options I am using:

Options.v().set_src_prec(Options.src_prec_apk);
Options.v().set_output_format(Options.output_format_none);

Options.v().set_no_bodies_for_excluded(true);
Options.v().set_allow_phantom_refs(true);
Options.v().set_android_jars(androidJar);

Options.v().set_process_dir(Collections.singletonList(apkPath));
Options.v().set_soot_classpath(androidJar);
Options.v().set_process_multiple_dex(true);

Options.v().set_whole_program(true);
Options.v().setPhaseOption("cg", "all-reachable:true");
Options.v().setPhaseOption("cg.spark", "on");
Options.v().setPhaseOption("cg.spark", "verbose:true");
Options.v().setPhaseOption("cg.spark", "string-constants:true");

Options.v().setPhaseOption("jb.ulp", "off");
 
Main.v().autoSetOptions();
Scene.v().loadNecessaryClasses();



I would greatly appreciate any pointers about what I might be missing (maybe an option that wasn't set properly) or what the problem could be.

Best regards,
Faridah Akinotcho

Eric Bodden

unread,
May 9, 2022, 8:19:44 AM5/9/22
to Faridah Akinotcho, Soot-list
Hi Faridah.

When creating callgraphs for Android you should be using our Soot-extension FlowDroid because it models Android’s lifecycle. When you use bare Soot then the callgraph will be very incomplete. Are you using FlowDroid already?

Cheers
Eric

_______________________________________________
Soot-list mailing list
Soot...@CS.McGill.CA
https://mailman.CS.McGill.CA/mailman/listinfo/soot-list

Faridah Akinotcho

unread,
May 9, 2022, 3:10:57 PM5/9/22
to Eric Bodden, Soot-list
Hi Prof. Dr. Bodden,

Thank you for the reply! I am using bare Soot because I am trying to add a wjtp subphase and I couldn't figure out how to do with Flowdroid without having to extend it and override constructCallgraph. Would there be a way for me to this with Flowdroid?

Best regards,
Faridah Akinotcho

Faridah Akinotcho

unread,
May 9, 2022, 10:27:14 PM5/9/22
to Eric Bodden, Arzt, Steven, Soot-list
Thank you for clarifying this! I'll try setting up my analysis within this Flowdroid callback instead.

Best regards,
Faridah Akinotcho


Le lundi 9 mai 2022, 12:12:00 UTC−7, Arzt, Steven <steve...@sit.fraunhofer.de> a écrit :


Dear Faridah,

You can call SetupApplication.addPreprocessor() to add a callback that gets invoked once the callgraph construction is done in FlowDroid. The interface has two methods, one before and one after callgraph construction. You just need to use the FlowDroid API, i.e., the SetupApplication class. That should be fairly simple.

Best regards,
  Steven


-----Original Message-----
From: Soot-list <soot-lis...@CS.McGill.CA> On Behalf Of Faridah Akinotcho
Sent: Montag, 9. Mai 2022 21:11
To: Eric Bodden <eric....@uni-paderborn.de>
Cc: Soot-list <soot...@cs.mcgill.ca>
Subject: Re: [Soot-list] Missing callgraph edges with Spark

Hi Prof. Dr. Bodden,

Thank you for the reply! I am using bare Soot because I am trying to add a wjtp subphase and I couldn't figure out how to do with Flowdroid without having to extend it and override constructCallgraph. Would there be a way for me to this with Flowdroid?

Best regards,
Faridah Akinotcho

Le lundi 9 mai 2022, 05:20:22 UTC−7, Eric Bodden <eric....@uni-paderborn.de> a écrit :


Hi Faridah.


When creating callgraphs for Android you should be using our Soot-extension FlowDroid because it models Android’s lifecycle. When you use bare Soot then the callgraph will be very incomplete. Are you using FlowDroid already?


Cheers
Eric



Faridah Akinotcho

unread,
May 10, 2022, 3:27:33 PM5/10/22
to Eric Bodden, Arzt, Steven, Soot-list
Hi Prof. Dr. Arzt, Prof. Dr. Bodden,

I followed your recommendations of using Flowdroid and preprocessors, instead of Soot, for which I thank you once again.

While it did provide me with a much more precise callgraph on the example I showed before, some of the edges that were found previously by Soot are now missing, and from a first look, it seems those are all within fragment lifecycles (e.g onViewCreated). Is there anything regarding how fragment classes are handled by Flowdroid which you believe could explain this behavior?

Best regards,
Faridah Akinotcho 

Faridah Akinotcho

unread,
May 11, 2022, 4:58:48 PM5/11/22
to Eric Bodden, Arzt, Steven, Soot-list
Dear Prof. Dr. Artz,

Thank you the clarifications. I noticed when looking into the code, the project I am building on top of is using Flowdroid 2.7.1, so I figured I should first update it to use the latest release and check if I encounter the same problem. 

However, it was a breaking change due to api changes, in particular some accessors being removed or fields becoming private (e.g getManifest(), getCallbackMethods() or getLayoutFileParser() were all removed from SetupApplication). I was wondering if there was any particular reason for this change, and if the only workaround would be to extend Flowdroid directly instead of using a packaged version.

Best regards,
Faridah Akinotcho

Le mardi 10 mai 2022, 12:31:51 UTC−7, Arzt, Steven <steve...@sit.fraunhofer.de> a écrit :


Dear Faridah,

FlowDroid applies a precise model for app components including fragments. You may want to look into the "AndroidEntryPointCreator" to identify any missing parts. Maybe there is a bug. Feel free to open merge request if you find something.
Le lundi 9 mai 2022, 05:20:22 UTC−7, Eric Bodden <eric....@uni-paderborn.de <mailto:eric....@uni-paderborn.de> > a écrit :


Hi Faridah.


When creating callgraphs for Android you should be using our Soot-extension FlowDroid because it models Android’s lifecycle. When you use bare Soot then the callgraph will be very incomplete. Are you using FlowDroid already?


Cheers
Eric



Faridah Akinotcho

unread,
May 12, 2022, 5:06:13 PM5/12/22
to Eric Bodden, Arzt, Steven, Soot-list
Hi again,

Please disregard my previous emails, It seems the API changes I mentionned were introduced by the developper of the project I'm working with and are unrelated to Flowdroid itself. Thanks again for all the information you provided.

Best regards,
Faridah Akinotcho

Faridah Akinotcho

unread,
May 13, 2022, 3:05:50 PM5/13/22
to Eric Bodden, Arzt, Steven, Soot-list
Hello Prof. Dr. Arzt, Prof.Dr.Bodden,

I reproduced the issue of edges missing for fragment lifecycles using Flowdroid 2.11-SNAPSHOT as well. 
Looking into AndroidEntryPointCreator, as you recommended, it seems method active bodies (and thus outgoing edges), will only be resolved for added fragments (i.e for which transactions and the correct type were identified by the CallbackAnalyzer ) but not for any other fragment class i.e classes within the app's namespace which extend android.app.Fragment or Support Library fragments.

Is this assumption correct? 

If that is the case, I guess one would either need to refine the fragment identifcation for those missing fragments or somehow modify the analysis to resolve active bodies for any fragment lifecycle as long as the fragment is within the app package.

Best regards,
Faridah Akinotcho
Le mardi 10 mai 2022, 12:31:51 UTC−7, Arzt, Steven <steve...@sit.fraunhofer.de> a écrit :


Dear Faridah,

FlowDroid applies a precise model for app components including fragments. You may want to look into the "AndroidEntryPointCreator" to identify any missing parts. Maybe there is a bug. Feel free to open merge request if you find something.

Best regards,
  Steven


-----Original Message-----
From: Faridah Akinotcho <faridath....@yahoo.fr>
Sent: Dienstag, 10. Mai 2022 21:28
To: Eric Bodden <eric....@uni-paderborn.de>; Arzt, Steven <steve...@sit.fraunhofer.de>
Cc: Soot-list <soot...@cs.mcgill.ca>
Subject: Re: [Soot-list] Missing callgraph edges with Spark

Hi Prof. Dr. Arzt, Prof. Dr. Bodden,

I followed your recommendations of using Flowdroid and preprocessors, instead of Soot, for which I thank you once again.

While it did provide me with a much more precise callgraph on the example I showed before, some of the edges that were found previously by Soot are now missing, and from a first look, it seems those are all within fragment lifecycles (e.g onViewCreated). Is there anything regarding how fragment classes are handled by Flowdroid which you believe could explain this behavior?

Best regards,
Faridah Akinotcho

Le lundi 9 mai 2022, 19:27:11 UTC−7, Faridah Akinotcho <faridath....@yahoo.fr> a écrit :


Thank you for clarifying this! I'll try setting up my analysis within this Flowdroid callback instead.


Best regards,
Faridah Akinotcho


Le lundi 9 mai 2022, 12:12:00 UTC−7, Arzt, Steven <steve...@sit.fraunhofer.de> a écrit :


Dear Faridah,

You can call SetupApplication.addPreprocessor() to add a callback that gets invoked once the callgraph construction is done in FlowDroid. The interface has two methods, one before and one after callgraph construction. You just need to use the FlowDroid API, i.e., the SetupApplication class. That should be fairly simple.

Best regards,
  Steven

-----Original Message-----
From: Soot-list <soot-lis...@CS.McGill.CA <mailto:soot-lis...@CS.McGill.CA> > On Behalf Of Faridah Akinotcho
Sent: Montag, 9. Mai 2022 21:11
Cc: Soot-list <soot...@cs.mcgill.ca <mailto:soot...@cs.mcgill.ca> >
Subject: Re: [Soot-list] Missing callgraph edges with Spark

Hi Prof. Dr. Bodden,

Thank you for the reply! I am using bare Soot because I am trying to add a wjtp subphase and I couldn't figure out how to do with Flowdroid without having to extend it and override constructCallgraph. Would there be a way for me to this with Flowdroid?

Best regards,
Faridah Akinotcho

Le lundi 9 mai 2022, 05:20:22 UTC−7, Eric Bodden <eric....@uni-paderborn.de <mailto:eric....@uni-paderborn.de> > a écrit :


Hi Faridah.


When creating callgraphs for Android you should be using our Soot-extension FlowDroid because it models Android’s lifecycle. When you use bare Soot then the callgraph will be very incomplete. Are you using FlowDroid already?


Cheers
Eric



Faridah Akinotcho

unread,
May 13, 2022, 5:52:58 PM5/13/22
to Eric Bodden, Arzt, Steven, Soot-list
Dear Prof. Dr. Arzt

Regarding the solutions you mentioned:

b) Implement an imprecise mode that picks up all fragments, regardless of whether they are used or not. This would, however, increase the size and complexity of the dummy main method, because you would need to associate each fragment with each activity.
> Yes, I agree this might be too imprecise. I was thinking of possibly treating all fragments as independent components (e.g like activities) for which  lifecycles would be modelled resolved by default, then only when used by some activity, they would be integrated within that activity lifecycle. 
But this might defeat the purpose of having the dummy main as a single entrypoint, so I'll focus on option a) instead.


> a) Find the code or XML that references the fragment and check why FlowDroid doesn't recognize it. Extend FlowDroid to detect this reference.

I started looking into the fragment identification to figure out what was going wrong on my example. It seems the issue occurs when resolving the type of the fragment in AbstractCallbackAnalyzer.analyzeMethodForFragmentTransaction
Value br = stmt.getInvokeExpr().getArg(i);

// Is this a fragment?
if (br.getType() instanceof RefType) {
   RefType rt = (RefType) br.getType();
Here, only the static type of the fragment is retained, independently from the context, which will default to parameter types for method invocations. 
For e.g:

void createFragment(android.app.Fragment fragment){
    fragmentManager.beginTransaction().add(fragment)
}

might be invoked with different types of fragments, but only android.app.Fragment will be returned here. 

I thought of using PointsToAnalysis.reachingObjects, then add all possible types as fragments of that activity. However, the pointer analysis is only set after the callgraph is constructed and is still a DumbPointerAnalysis in this stage.

Thus, I think a solution would either be to set the pointer analysis earlier than when applying the "cg" pack or to perform a backward data-flow analysis for definitions of the fragment object. 
Would you recommend one over the other?


Best regards,
Faridah Akinotcho

Le vendredi 13 mai 2022, 14:14:15 UTC−7, Arzt, Steven <steve...@sit.fraunhofer.de> a écrit :


Dear Faridah,

FlowDroid integrates fragments into its dummy main method if it detects that the fragment is actually used in some activity. This allows FlowDroid to precisely associate fragments with their respective parent activity. Therefore, if the fragment is there, but FlowDroid cannot find code that instantiates it, or XML files that reference it, the fragment is missed. I see two possible solutions:

a) Find the code or XML that references the fragment and check why FlowDroid doesn't recognize it. Extend FlowDroid to detect this reference.

b) Implement an imprecise mode that picks up all fragments, regardless of whether they are used or not. This would, however, increase the size and complexity of the dummy main method, because you would need to associate each fragment with each activity.
Le lundi 9 mai 2022, 19:27:11 UTC−7, Faridah Akinotcho <faridath....@yahoo.fr <mailto:faridath....@yahoo.fr> > a écrit :


Thank you for clarifying this! I'll try setting up my analysis within this Flowdroid callback instead.


Best regards,
Faridah Akinotcho


Le lundi 9 mai 2022, 12:12:00 UTC−7, Arzt, Steven <steve...@sit.fraunhofer.de <mailto:steve...@sit.fraunhofer.de> > a écrit :


Dear Faridah,

You can call SetupApplication.addPreprocessor() to add a callback that gets invoked once the callgraph construction is done in FlowDroid. The interface has two methods, one before and one after callgraph construction. You just need to use the FlowDroid API, i.e., the SetupApplication class. That should be fairly simple.

Best regards,
  Steven

-----Original Message-----
From: Soot-list <soot-lis...@CS.McGill.CA <mailto:soot-lis...@CS.McGill.CA>  <mailto:soot-lis...@CS.McGill.CA <mailto:soot-lis...@CS.McGill.CA> > > On Behalf Of Faridah Akinotcho
Sent: Montag, 9. Mai 2022 21:11
To: Eric Bodden <eric....@uni-paderborn.de <mailto:eric....@uni-paderborn.de>  <mailto:eric....@uni-paderborn.de <mailto:eric....@uni-paderborn.de> > >
Cc: Soot-list <soot...@cs.mcgill.ca <mailto:soot...@cs.mcgill.ca>  <mailto:soot...@cs.mcgill.ca <mailto:soot...@cs.mcgill.ca> > >
Subject: Re: [Soot-list] Missing callgraph edges with Spark

Hi Prof. Dr. Bodden,

Thank you for the reply! I am using bare Soot because I am trying to add a wjtp subphase and I couldn't figure out how to do with Flowdroid without having to extend it and override constructCallgraph. Would there be a way for me to this with Flowdroid?

Best regards,
Faridah Akinotcho

Le lundi 9 mai 2022, 05:20:22 UTC−7, Eric Bodden <eric....@uni-paderborn.de <mailto:eric....@uni-paderborn.de>  <mailto:eric....@uni-paderborn.de <mailto:eric....@uni-paderborn.de> > > a écrit :


Hi Faridah.


When creating callgraphs for Android you should be using our Soot-extension FlowDroid because it models Android’s lifecycle. When you use bare Soot then the callgraph will be very incomplete. Are you using FlowDroid already?


Cheers
Eric



Reply all
Reply to author
Forward
0 new messages