Jira (FACT-2551) Resolver is called multiple times even if the fact cannot be resolved

[Gheorghe Popescu (Jira)]

Gheorghe Popescu created an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved Issue Type: Improvement Assignee: Unassigned Created: 2020/04/14 2:23 AM Priority: Normal Reporter: Gheorghe Popescu This can have major performance implications if the resolver is slow.   Context: Facter 4 is caching resolver data so that resolver that are used by multiple facts to make the system calls only once.   Issue: If a fact cannot be retrieved from the system, the resolver is called each time that fact is request, and all the system calls are done again.   Example:  OS X system_profiler : https://github.com/puppetlabs/facter-ng/blob/master/lib/resolvers/macosx/system_profiler_resolver.rb#L38 does not have information about `subsystem_vendor_id` OS X is_virtual is calling again for the `subsystem_vendor_id`: https://github.com/puppetlabs/facter-ng/blob/975344c6a0bdde9bbc208ff6381c00733a48e0c0/lib/facts/macosx/is_virtual.rb#L25   This makes the resolver to make 2 expensive system calls of ~0.4 seconds each(2/3 of total facter run)   Backtrace: ❯ bx facter------------------------------------------First Call fact_list: {} fact_name: model_identifier   user system total real 0.031205 0.021051 0.291823 ( 0.438509) -------------------------------------------------- calling retrieve_system_profiler ["facter-ng/lib/resolvers/macosx/system_profiler_resolver.rb:40:in `block in post_resolve'", "facter-ng/lib/resolvers/macosx/system_profiler_resolver.rb:40:in `fetch'", "facter-ng/lib/resolvers/macosx/system_profiler_resolver.rb:40:in `post_resolve'", "facter-ng/lib/resolvers/base_resolver.rb:21:in `block in resolve'", "facter-ng/lib/resolvers/base_resolver.rb:19:in `synchronize'", "facter-ng/lib/resolvers/base_resolver.rb:19:in `resolve'", "facter-ng/lib/facts/macosx/is_virtual.rb:19:in `hypervisor_name'", "facter-ng/lib/facts/macosx/is_virtual.rb:15:in `virtual?'", "facter-ng/lib/facts/macosx/is_virtual.rb:9:in `call_the_resolver'", "facter-ng/lib/framework/core/fact/internal/core_fact.rb:12:in `create'", "facter-ng/lib/framework/core/fact/internal/internal_fact_manager.rb:27:in `block (2 levels) in start_threads'"] -------------------------------------------------- Second Call   fact_list: { :model_name=> 'value' :model_identifier=> 'value' :processor_name=> 'value' :processor_speed=> 'value' :number_of_processors=> 'value' :total_number_of_cores=> 'value' :l2_cache_per_core=> 'value' :l3_cache=> 'value' :"hyper-threading_technology"=> 'value' :memory=> 'value' :boot_rom_version=> 'value' :smc_version_system=> 'value' :serial_number_system=> 'value' :hardware_uuid=> 'value' :system_version=> 'value' :kernel_version=> 'value' :boot_volume=> 'value' :boot_mode=> 'value' :computer_name=> 'value' :user_name=> 'value' :secure_virtual_memory=> 'value' :system_integrity_protection=> 'value' :time_since_boot=> 'value' } fact_name: subsystem_vendor_id     user system total real 0.000297 0.001640 0.209305 ( 0.430579) -------------------------------------------------- calling retrieve_system_profiler ["facter-ng/lib/resolvers/macosx/system_profiler_resolver.rb:40:in `block in post_resolve'", "facter-ng/lib/resolvers/macosx/system_profiler_resolver.rb:40:in `fetch'", "facter-ng/lib/resolvers/macosx/system_profiler_resolver.rb:40:in `post_resolve'", "facter-ng/lib/resolvers/base_resolver.rb:21:in `block in resolve'", "facter-ng/lib/resolvers/base_resolver.rb:19:in `synchronize'", "facter-ng/lib/resolvers/base_resolver.rb:19:in `resolve'", "facter-ng/lib/facts/macosx/is_virtual.rb:25:in `hypervisor_name'", "facter-ng/lib/facts/macosx/is_virtual.rb:15:in `virtual?'", "facter-ng/lib/facts/macosx/is_virtual.rb:9:in `call_the_resolver'", "facter-ng/lib/framework/core/fact/internal/core_fact.rb:12:in `create'", "facter-ng/lib/framework/core/fact/internal/internal_fact_manager.rb:27:in `block (2 levels) in start_threads'"] --------------------------------------------------       Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Gheorghe Popescu (Jira)]

Gheorghe Popescu updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved

Change By: Gheorghe Popescu

This can have major performance implications if the resolver is slow.

* Context: *

Facter 4 is caching resolver data so that resolver that are used by multiple facts to make the system calls only once.

* Issue: *

If a fact cannot be retrieved from the system, the resolver is called each time that fact is request, and all the system calls are done again.

* Example: *

OS X system_profiler : [https://github.com/puppetlabs/facter-ng/blob/master/lib/resolvers/macosx/system_profiler_resolver.rb#L38] does not have information about `subsystem_vendor_id` OS X is_virtual is calling again for the `subsystem_vendor_id`: [https://github.com/puppetlabs/facter-ng/blob/975344c6a0bdde9bbc208ff6381c00733a48e0c0/lib/facts/macosx/is_virtual.rb#L25]   This makes the resolver to make 2 expensive system calls of ~0.4 seconds each(2/3 of total facter run)

* Backtrace: * {code:java}❯ bx facter------------------------------------------First Call

{code}

Add Comment

[Gheorghe Popescu (Jira)]

Gheorghe Popescu updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved Change By: Gheorghe Popescu

This can have major performance implications if the resolver is slow.

*Context:* Facter 4 is caching resolver data so that resolvers that are used by multiple facts make the system calls only once. *Issue:* If a fact cannot be retrieved from the system, the resolver is called each time that fact is request requested , and all the system calls are done again.  *Example:*

OS X system_profiler : [https://github.com/puppetlabs/facter-ng/blob/master/lib/resolvers/macosx/system_profiler_resolver.rb#L38] does not have information about `subsystem_vendor_id` OS X is_virtual is calling again for the `subsystem_vendor_id`: [https://github.com/puppetlabs/facter-ng/blob/975344c6a0bdde9bbc208ff6381c00733a48e0c0/lib/facts/macosx/is_virtual.rb#L25]   This makes the resolver to make 2 expensive system calls of ~0.4 seconds each(2/3 of total facter run)

*Backtrace:*

[Gheorghe Popescu (Jira)]

Gheorghe Popescu updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved Change By: Gheorghe Popescu

This can have major performance implications if the resolver is slow. *Context:*

Facter 4 is caching resolver data so that resolver resolvers that are used by multiple facts to make the system calls only once. *Issue:* If a fact cannot be retrieved from the system, the resolver is called each time that fact is request, and all the system calls are done again.

[Gheorghe Popescu (Jira)]

Gheorghe Popescu updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved Change By: Gheorghe Popescu

This can have major performance implications if the resolver is slow. *Context:*

Facter 4 is caching resolver data so that resolvers that are used by multiple facts make the system calls only once. *Issue:* If a fact cannot be retrieved from the system, the resolver is called each time that fact is requested, and all the system calls are done again.

*Proposed solution:*   system call could be memoized.

Add Comment

[Gheorghe Popescu (Jira)]

Gheorghe Popescu updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved

Change By: Gheorghe Popescu Epic Link: FACT-2123

Add Comment

[Mihai Buzgau (Jira)]

Mihai Buzgau commented on FACT-2551

Re: Resolver is called multiple times even if the fact cannot be resolved Base resolver should set nil on facts that cannot be resolved.

Add Comment

[Mihai Buzgau (Jira)]

Mihai Buzgau updated an issue

Facter / FACT-2551

Resolver is called multiple times even if the fact cannot be resolved

Change By: Mihai Buzgau Sprint: ready for triage

Add Comment

[Mihai Buzgau (Jira)]

Mihai Buzgau updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved

Change By: Mihai Buzgau Team: Night's Watch

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved

Change By: Bogdan Irimie Sprint: ready for triage ghost-26.08

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved

Change By: Bogdan Irimie Story Points: 2

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved

Change By: Bogdan Irimie Sprint: ghost-26.08 , ready for triage 2

Add Comment

[Sebastian Miclea (Jira)]

Sebastian Miclea assigned an issue to Sebastian Miclea

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved

Change By: Sebastian Miclea Assignee: Sebastian Miclea

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

Facter / FACT-2551 Resolver is called multiple times even if the fact cannot be resolved

Change By: Bogdan Irimie Sprint: ghost-26.08, ghost-2.09 , ready for triage 2

Add Comment