De-serialization memory leak

[Saeed Fathollahzadeh]

Hello everyone,

I try to bench multiple serialization methods and I used Kryo for the benchmark. Our benchmark wants to sort a large amount of data with a small machine, some we need to divide data into multiple chunks. After sorting each chunk we collect and generate the last results. During these phases, the Kryo faced with a memory leak problem. I enclosed the snapshot of my memory for more info. 

Is there anyone to help me, how can I fix the problem?

Regard,

Saeed

[Joachim Durchholz]

Hi Saeed,

that's too vague to give you any useful response.
No source code, no stripped-down source code, no comparison of expected
vs. actual problem, the screenshot highlights a line that seems
unrelated to Kryo - we have no chance.

A pretty concise and general list of instructions for a bug report we
can answer to can be found on http://sscce.org/.

HTH
Jo

Am 02.07.20 um 10:59 schrieb Saeed Fathollahzadeh:

> --
> You received this message because you are subscribed to the "kryo-users"
> group.
> http://groups.google.com/group/kryo-users
> ---
> You received this message because you are subscribed to the Google
> Groups "kryo-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kryo-users+...@googlegroups.com
> <mailto:kryo-users+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kryo-users/cb256407-fa2c-42ad-86f0-f6576565c3d6o%40googlegroups.com
> <https://groups.google.com/d/msgid/kryo-users/cb256407-fa2c-42ad-86f0-f6576565c3d6o%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Saeed Fathollahzadeh]

Dear Joachim,

Thanks for the answer. For more clearly explain my problem, I enclosed some sections of the source code.

// >>> Root Class:

private String created_at;

private long id;

private String text;

private String source;

private boolean truncated;

private long in_reply_to_status_id;//nullable

private long in_reply_to_user_id;//nullable

private String in_reply_to_screen_name;//nullable

private User user;

private Coordinates coordinates;//nullable

private Place place;//nullable

private long quoted_status_id;

private boolean is_quote_status;

private TweetStatus quoted_status;

private TweetStatus retweeted_status;

private int quote_count;//nullable

private int reply_count;

private int retweet_count;

private int favorite_count;//nullable

private Entities entities;

private ExtendedEntities extended_entities;

private boolean favorited;//nullable

private boolean retweeted;

private boolean possibly_sensitive;//nullable

private String filter_level;

private String lang;//nullable

private List<MatchingRulesEntity> matching_rules;

private long current_user_retweet;//nullable

private Map<String, Boolean> scopes;//nullable

private boolean withheld_copyright;//nullable

private List<String> withheld_in_countries;//nullable

private String withheld_scope;//nullable

//>>>>> User Class:

private long id;

private String name;

private String screen_name;

private String location;//nullable

private String url;//nullable

private String description;//nullable

//TODO: protected is a java keyboard so we need a solution for get data from text string

private boolean isProtected;

private boolean verified;

private int followers_count;

private int friends_count;

private int listed_count;

private int favourites_count;

private int statuses_count;

private String created_at;

private String profile_banner_url;

private String profile_image_url_https;

private boolean default_profile;

private List<String> withheld_in_countries;

private String withheld_scope;

//No longer supported (deprecated) attributes

private List<URLEntity> descriptionURLEntities;

private boolean geo_enabled;

private String lang;

private boolean contributors_enabled;

private String profile_background_color;

private String profile_background_image_url;

private String profile_background_image_url_https;

private boolean profile_background_tile;

private String profile_image_url;

private String profile_link_color;

private String profile_sidebar_border_color;

private String profile_sidebar_fill_color;

private String profile_text_color;

private boolean profile_use_background_image;

private int utc_offset;

private String time_zone;

private boolean is_translator;

private boolean follow_request_sent;

private boolean showAllInlineMedia;

//>>> Place Class:

private String name;

    private String country_code;

    private String id;

    private String country;

    private String place_type;

    private String url;

    private String full_name;

    private BoundingBoxCoordinate bounding_box;

=========================== There are more classes in the project. 

and my Singleton Kryo class:

kryo = new Kryo(); kryo.register(LinkedHashMap.class); kryo.register(ArrayList.class); kryo.register(HashMap.class); kryo.register(double[].class); kryo.register(com.google.gson.internal.LinkedTreeMap.class); kryo.register(AdditionalMediaInfoEntity.class); kryo.register(BoundingBoxCoordinate.class); kryo.register(Coordinates.class); kryo.register(Entities.class); kryo.register(ExtendedEntities.class); kryo.register(HashtagEntity.class); kryo.register(MatchingRulesEntity.class); kryo.register(MediaEntity.class); kryo.register(MediaSizesEntity.class); kryo.register(OptionEntity.class); kryo.register(Place.class); kryo.register(PollEntity.class); kryo.register(SizeEntity.class); kryo.register(SymbolEntity.class); kryo.register(TweetStatus.class); kryo.register(URLEntity.class); kryo.register(User.class); kryo.register(UserMentionEntity.class); kryo.register(VariantEntity.class); kryo.register(VideoEntity.class); kryo.register(AdditionalMediaInfoEntityFBS.class); kryo.register(BoundingBoxCoordinateFBS.class); kryo.register(CoordinatesFBS.class); kryo.register(EntitiesFBS.class); kryo.register(ExtendedEntitiesFBS.class); kryo.register(HashtagEntityFBS.class); kryo.register(MatchingRulesEntityFBS.class); kryo.register(MediaEntityFBS.class); kryo.register(MediaSizesEntityFBS.class); kryo.register(OptionEntityFBS.class); kryo.register(PlaceFBS.class); kryo.register(PollEntityFBS.class); kryo.register(SizeEntityFBS.class); kryo.register(SymbolEntityFBS.class); kryo.register(TweetStatusFBS.class); kryo.register(URLEntityFBS.class); kryo.register(UserFBS.class); kryo.register(UserMentionEntityFBS.class); kryo.register(VariantEntityFBS.class); kryo.register(VideoEntityFBS.class); kryo.register(TweetStatusProtos.TweetStatusP.class); //>>>> Kryo deserialization function: public RootData kryoDeserialization(byte[] buf, Class<?> myclass, Kryo kryo) { ByteArrayInputStream byteInputStream = new ByteArrayInputStream(buf); Input input = new Input(byteInputStream); return (RootData) kryo.readObject(input, myclass); } Let me know if you need more info. Thanks, Saeed

> an email to kryo-...@googlegroups.com
> <mailto:kryo-users+unsub...@googlegroups.com>.

[Joachim Durchholz]

That's not an SSCCE.

Am 02.07.20 um 20:14 schrieb Saeed Fathollahzadeh:

> > <mailto:kryo-users+...@googlegroups.com>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kryo-users/cb256407-fa2c-42ad-86f0-f6576565c3d6o%40googlegroups.com
> <https://groups.google.com/d/msgid/kryo-users/cb256407-fa2c-42ad-86f0-f6576565c3d6o%40googlegroups.com>
>
> >

> <https://groups.google.com/d/msgid/kryo-users/cb256407-fa2c-42ad-86f0-f6576565c3d6o%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kryo-users/cb256407-fa2c-42ad-86f0-f6576565c3d6o%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the "kryo-users"
> group.
> http://groups.google.com/group/kryo-users
> ---
> You received this message because you are subscribed to the Google
> Groups "kryo-users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to kryo-users+...@googlegroups.com
> <mailto:kryo-users+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/kryo-users/b4a37cb3-0c87-4db0-8616-e9718141b952o%40googlegroups.com
> <https://groups.google.com/d/msgid/kryo-users/b4a37cb3-0c87-4db0-8616-e9718141b952o%40googlegroups.com?utm_medium=email&utm_source=footer>.