Session Partitioning and Lookup caching

20 views
Skip to first unread message

Sats

unread,
Jul 16, 2010, 3:07:13 PM7/16/10
to denver-infor...@googlegroups.com
I ve put in 12 partitions on a mapping, with SQL overrides at the session level. What I wanted to know was if the lookups in the table will be cached 12 times?
I m attaching this session log, where it seems to be indicating that it is caching the lookups 12 times. But when we go check the cache folder we dont see as many files.
How do we know how much memory the workflow is using?


Sats
s_m_PCM_drivers_QDNA_SC_DRIVER_FACT_log.txt

Sats

unread,
Jul 16, 2010, 4:04:01 PM7/16/10
to denver-infor...@googlegroups.com
This is the part of the log file that confuses me. It repeats the following message 12 times for every lookup in the mapping. What does it mean?

Severity    Timestamp    Node    Thread    Message Code    Message
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes
INFO    7/15/2010 2:13:20 PM    node_01    MAPPING    CMN_1793    The index cache size that would hold [2789753] rows in the lookup table for [lkp_sc_customer_dim], in memory, is [353015600] bytes

Tom Nats

unread,
Jul 16, 2010, 4:12:00 PM7/16/10
to denver-infor...@googlegroups.com
2010-07-15 14:13:21 : INFO : (28723 | MANAGER) : (IS |
IS_86_ASCII_GRID) : node_01 : PETL_24005 : PETL_24005 Starting
post-session tasks. : (Thu Jul 15 14:13:21 2010)
2010-07-15 14:13:21 : INFO : (28723 | MANAGER) : (IS |
IS_86_ASCII_GRID) : node_01 : PETL_24029 : PETL_24029 Post-session
task completed successfully. : (Thu Jul 15 14:13:21 2010)
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_17_0_477891L64] for transformation
[Shortcut_to_r_lkp_ETL_PROC_CNTRL_ID_SEQ_nextval_unconnected].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_38_0_477891L64] for transformation
[lkp_sc_prd_asgnd_pne_combo_dim].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_4_0_477891L64] for transformation [lkp_sc_customer_dim].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_14_0_477891L64] for transformation
[lkp_sc_source_system_dim].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_16_0_477891L64] for transformation
[Shortcut_to_r_lkp_ETL_PROCESS_ID_unconnected].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_19_0_477891L64] for transformation
[Shortcut_to_r_lkp_ETL_PROCESS_CONTROL_ID_PMsessionname_unconnected].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_37_0_477891L64] for transformation [lkp_sc_metrics_dim].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_15_0_477891L64] for transformation
[Shortcut_to_r_lkp_DUAL_etl_utc_date_unconnected].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TE_7216 : Deleting cache files
[PMLKUP31154_5_0_477891L64] for transformation [lkp_sc_location_dim].
2010-07-15 14:13:21 : INFO : (28723 | MAPPING) : (IS |
IS_86_ASCII_GRID) : node_01 : TM_6018 : The session completed with [0]
row transformation errors.
2010-07-15 14:13:21 : INFO : (28723 | MANAGER) : (IS |
IS_86_ASCII_GRID) : node_01 : PETL_24002 : Parallel Pipeline Engine
finished.
2010-07-15 14:13:21 : INFO : (28723 | DIRECTOR) : (IS |
IS_86_ASCII_GRID) : node_01 : PETL_24012 : Session run completed
successfully.
2010-07-15 14:13:21 : INFO : (28723 | DIRECTOR) : (IS |
IS_86_ASCII_GRID) : node_01 : TM_6022 :
Reply all
Reply to author
Forward
0 new messages