Seeking Methodological Guidance: Comparative Testing in Within-Sample Parallel PLS-SEM Models

[Nikita Singh Dahiya]

Respected Neeraj Sir and Group members,

I am a doctoral researcher seeking methodological guidance on my PhD thesis, which examines and compares the effectiveness of Human Influencer Characteristics (HIC) and Virtual Influencer Characteristics (VIC) in generating Emotional Engagement, Perceived Credibility, and Consumer Behaviour among Indian social media users.

STUDY DESIGN
The study employs a within-sample parallel design — the same respondents evaluated both HIC and VIC through two structurally equivalent questionnaire modules. Both constructs are specified as reflective-formative higher-order constructs within PLS-SEM (SmartPLS 4.0) using the disjoint two-stage approach. The model includes Emotional Engagement and Perceived Credibility as mediators between HIC/VIC and Consumer Behaviour.

COMPARATIVE HYPOTHESES
H4a: HIC generates higher Emotional Engagement than VIC.
H4b: HIC generates higher Perceived Credibility than VIC.
H4c: HIC exerts a stronger direct effect on Consumer Behaviour than VIC.

METHODOLOGICAL CHALLENGE
I wish to formally test whether the differences in path coefficients across the two parallel models are statistically significant. However, the within-sample design violates the independence assumption of conventional PLS Multi-Group Analysis (PLS-MGA), which requires independent respondent groups (Hair et al., 2019).

I have identified three possible approaches and seek your expert opinion:

1. ADAPTED PERMUTATION-BASED MGA: Söllner, Mishra, Becker, and Leimeister (2024, European Journal of Information Systems) extended the standard permutation approach to handle dependent/repeated-measures samples in PLS-SEM. Could this adaptation apply to a within-sample cross-sectional parallel design, where dependence arises from the same respondents rating two conditions simultaneously rather than across longitudinal time waves?

2. PAIRED T-TEST ON COMPOSITE SCORES: Dondapati and Dehury (2024, Computers in Human Behavior: Artificial Humans) used a within-subject HI vs. VI design with Indian respondents and applied paired t-tests on composite scores. Would this be a valid and sufficient approach for formally testing H4a, H4b, and H4c in a doctoral thesis?

3. DESCRIPTIVE COMPARISON ONLY: Several published parallel within-sample HI vs. VI studies — including Guardia et al. (2025), Schouten et al. (2020), and Kim and Im (2025) — support comparative hypotheses through path coefficient and effect size differentials alone, citing the within-sample design as a limitation. Is this approach methodologically acceptable for a PhD thesis with an explicit comparative objective?

SPECIFIC QUESTIONS
1. Is the Söllner et al. (2024) adapted permutation approach applicable to my cross-sectional within-sample design, or is it specific to longitudinal repeated-measures structures?
2. If not, what is the most rigorous formal test available for comparing path coefficients across two dependent parallel PLS-SEM models?
3. Is a descriptive comparison of path coefficients and effect sizes — without a formal difference test — sufficient for a doctoral thesis with explicit comparative hypotheses?

I'd appreciate any guidance or relevant references the group can share.

Thank you for your time.

Nikita
Ph.D. Scholar
SRM University

[Neeraj Kaushik]

Dear Nikita 

As per your post, you've

• one IDV Influencer characteristic: Human or Virtual
• two mediators: Emotional Engagement and Perceived Credibility and
• one DV Consumer Behaviour

You'll manipulate IDV experimentally and measure the mediators and DV.
Now before answering your questions, I've an observation about your research design.

You wrote

The study employs a within-sample parallel design: the same respondents evaluated both HIC and VIC through two structurally equivalent questionnaire modules.

Now here is a question: when you show the different stimuli to same group (first Human and then Virtual, or the reverse), won't there be a bias because the audience already knows the context and questions?

Best wishes

Neeraj

--
The members of this group are expected to follow the following Protocols:
1. Please search previous posts in the group before posting the question.
2. Don't write the query in someone's post. Always use the option of New topic for the new question. You can do this by writing to dataanaly...@googlegroups.com
3. It’s better to give a proper subject to your post/query. It'll help others while searching.
4. Never write Open-ended queries. This group intends to help research scholars, NOT TO WORK FOR THEM.
5. Never write words like URGENT in your posts. People will help when they are free.
6. Never upload any information about National Seminars/Conferences. Send such information
in personal emails and feel free to share any RESEARCH-related information.
7. No Happy New Year, Happy Diwali, Happy Holi, Happy Birthday, Happy Anniversary, etc. allowed in this group.
8. Asking or sharing Research Papers is NOT ALLOWED.
9. You can share your questionnaire only once.
---
You received this message because you are subscribed to the Google Groups "DataAnalysis" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataanalysistrai...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dataanalysistraining/9453ded9-cb6e-478a-9aef-b4a8b526e298n%40googlegroups.com.

[Nikita Singh Dahiya]

Respected Neeraj Sir,

Thank you for raising this concern. I would like to clarify two aspects of my design that address the bias you mentioned.

First, the two sections are clearly separated with distinct labels and a brief explanation of virtual influencers before the VI section begins. Respondents were fully aware they were evaluating a different type of influencer.

Second, the scales are conceptually distinct — HIC measures Authenticity, Expertise, Reliability, and Relatability, while VIC measures Visual Appeal, Novelty, Anthropomorphic Cues, and Technological Sophistication. The low conceptual overlap between the two reduces the risk of one section contaminating the other.

I acknowledge that all respondents completed the HI section first, which remains a limitation. This is consistent with published within-sample comparative studies such as Dondapati and Dehury (2024) and Guardia et al. (2025), which followed a similar design and noted order effects as a limitation.


Thank you.

Warm regards,
Nikita
Ph.D. Scholar, SRM University Delhi-NCR

[Neeraj Kaushik]

Dear Nikita

If you are measuring different constructs from the virtual and human influencer groups, what will you compare?

Plz prepare an Excel file of dummy data for your variables and share it here.

Best wishes

To view this discussion visit https://groups.google.com/d/msgid/dataanalysistraining/0f903c23-7216-4afc-9d04-42fef93e6a4cn%40googlegroups.com.

[Nikita Singh Dahiya]

Respected Neeraj Sir,

Thank you for your question. Please find the dummy data file attached.

Regarding your observation about comparing different constructs, I would like to clarify that I am not directly comparing HIC with VIC. The comparison is between their effects on the same outcome variable, Consumer Behaviour (CB).

Specifically:
- HIC → CB
- VIC → CB

Both models use the same respondents and the same CB items (Attitude, Trust, and Purchase Intention), measured separately for each influencer type. So the basis of comparison is not the constructs themselves but their predictive power on an identical dependent variable.

Regarding the dummy data file: it contains 10 dummy respondents with values on a 5-point Likert scale. The file shows the complete variable structure, HI characteristics (16 items), VI characteristics (16 items), and the shared constructs, Emotional Engagement, Perceived Credibility, Attitude, Trust, and Purchase Intention, each measured separately for both HI and VI by the same respondents.

Thank you.

Warm regards,
Nikita
Ph.D. Scholar, SRM University Delhi-NCR

To view this discussion visit https://groups.google.com/d/msgid/dataanalysistraining/CAAd%3Dc8PHb9v0QHCy97uNop8bMHXO7thNz_UeKqzSi7STUYgi7Q%40mail.gmail.com.

[Neeraj Kaushik]

Dear NSD,

Since your independent variables (HIC and VIC) consist of entirely different sets of dimensions and constructs, comparing their individual path coefficients directly to Consumer Behaviour is not methodologically valid. To directly compare path coefficients across two models, the predictor constructs and their measurement models must be identical.

In your current setup, because the predictors are conceptualized and measured differently, any difference in path coefficients could be due to differences in the constructs themselves rather than a true difference in predictive power.

If your core objective is to compare which influencer type (Human vs. Virtual) has a stronger overall effect on Consumer Behaviour, you would need to either:

1.  Use an experimental design comparing two conditions using a common, higher-order construct or unified framework for both groups, or
2.  Compare the overall variance explained (R-square) of the dependent variable across the two separate models rather than comparing individual path coefficients.

Best wishes
Neeraj

To view this discussion visit https://groups.google.com/d/msgid/dataanalysistraining/CAL%3DHsbH2x8cVogXVij6G3Ecv1AzJGu04XaPWaG1%3DYdny0n-kdw%40mail.gmail.com.

[Nikita Singh Dahiya]

Respected Neeraj Sir,

Thank you for your guidance. I will address H4c through R-square and effect size comparison as suggested.

However, I would like to seek your advice on the remaining hypotheses under Objective 4:

Objective 4: To compare the effectiveness of HIC and VIC in generating Emotional Engagement, Perceived Credibility, and Consumer Behaviour.

H4a: HIC generates higher Emotional Engagement than VIC.
H4b: HIC generates higher Perceived Credibility than VIC.

H4c: HIC has a stronger direct effect on Consumer Behaviour than VIC.

For H4a and H4b, the situation is different. Emotional Engagement and Perceived Credibility are conceptually identical constructs measured separately for both HI and VI conditions using parallel equivalent scales. Unlike HIC and VIC, the mediators are the same constructs evaluated in two conditions.

Would a paired t-test on the latent variable scores of EE_HI vs EE_VI and PC_HI vs PC_VI be a valid approach to formally test H4a and H4b? Or should I compare R-square values of the mediators as I am doing for H4c?

Thank you.

Warm regards,
Nikita
Ph.D. Scholar, SRM University Delhi-NCR