Issue with Frequency Reuse Algorithms Evaluation (3GPP Reference Scenario use-case)

[Andrei Marinescu]

Hi,

I've been working for a while with ns-3 for a project with an industrial partner, and as part of the project's task we were meant to validate the ns-3 lte module in a set of reference 3GPP scenarios (in particular wrt Case 1 from TS 36.814). For this task I've used a modified version of the lena-dual-stripe.cc code (available on github here - needs to be renamed to lena-dual-stripe.cc for attached python code to work) to adapt it to the 3GPP requirements/parameters (if you're interested you can find more information about the validation process in an article here), as we needed to use the EPC and TCP/UDP functionalities of ns-3 in order to fully replicate the required scenario. As end result, we obtained results that quite closely resemble 3GPP values (which represent an aggregate of 17 industrial simulators) with regard to UE SINR and throughput.

This validation was only the initial starting point of our project, as its main focus is on frequency reuse algorithms (another feature that ns-3's LTE module has). After obtaining some rather counterintuitive results in terms of UE throughput performance, we went back to the validation scenario (3GPP Case 1 from TS 36.814). Here, it turns out that while the UE SINR for frequency reuse-based algorithms is better (e.g, see attached the SINR output for hard reuse vs full reuse in fig0.tif) , the actual obtained UE throughput is worse for frequency-reuse based algorithms (e.g., throughput output for hard reuse vs full reuse in fig1.tif), even for edge UEs (where frequency reuse should bring the highest gain). 

In essence, there is no point on the CDF we obtained (fig1.tif) where the hard reuse algorithm performs better than the full frequency reuse algorithm (note that these results are obtained with the proportional fair scheduler, which is the only scheduler compatible with frequency reuse algorithms in ns-3, as opposed to the validation results from the paper which were obtained with the round-robin scheduler). Basically the results show no tradeoff between coverage and throughput between hard reuse and full reuse schemes, so they contradict the literature (and in fact the entire purpose of frequency reuse), as at least some edge users should have better throughput in the hard frequency reuse case than in the full reuse one.

I was wondering if you could explain this behaviour, or point me towards the source of the potential error/bug in these simulations. I've also attached the python code we use to configure parameters and trigger simulations (this needs to be placed in the ns-3 root folder, same one were waf is run from). Note that some of the scenario parameters are configured directly in the modified version of lena-dual-stripe.cc.

Would really appreciate your help!
Andrei

[Andrei Marinescu]

Just realised I didn't mention the platform and ns-3 version used.

Platform: Ubuntu 16.04
Version: Initial work started with ns-3.24, issue has been replicated both on it and on ns-3.27

[Tommaso Pecorella]

Hi Andrei,

thanks for the detailed report. I guess that the only people able to dig in the problem are at CTTC.

I'd strongly suggest to involve Biljana Bojovic <bboj...@cttc.es> in the discussion.

I'll forward the message to her.

Cheers,

T.

[Biljana Bojovic]

Hi Andrei,

thanks for your great work on validation of ns-3 lte module!

Regarding your question, I tried to find which configuration of hard FFR are you using, but I didn't find any configuration neither in lena-dual-stripe neither in .py script. Can you provide more info on this?

The performance of FFR depends a lot on the configuration of the scenario, and normally are used some SON techniques to adjust its parameters. While ns-3 hard FFR algorithm provides the hard FFR functionality, it does not include this SON logic that would automatically find the best system configuration for FFR algorithm.

Cheers,

Biljana

--
Posting to this group should follow these guidelines https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
---
You received this message because you are subscribed to the Google Groups "ns-3-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ns-3-users+unsubscribe@googlegroups.com.
To post to this group, send email to ns-3-...@googlegroups.com.
Visit this group at https://groups.google.com/group/ns-3-users.
For more options, visit https://groups.google.com/d/optout.

[Andrei Marinescu]

Many thanks for getting back to me Biljana and Tommasso!

Biljana, I haven't used any SON technique for the hard FFR scenario. However, the purpose of our project is to use machine learning techniques to automatically configure the cells/network parameters employing Strict Frequency Reuse (where we adjust bandwidth, the RSRQ threshold that defines center users, and power levels for edge/center bandwidths) depending on network conditions. As the results we were obtaining weren't conclusive, we decided to go back to basics and test the more simple hard frequency reuse algorithm in the already validated scenarios. 

To my understanding, hard FFR divides the DL bandwidth evenly (more or less, depending on the total number of RBs available for DL)  between the three sectors of a site, in an orthogonal manner. Would you be able to explain how SON could assist this process, or eventually lead me to some existing examples/implementations in ns-3 for this?

Cheers,
Andrei

You received this message because you are subscribed to a topic in the Google Groups "ns-3-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ns-3-users/3K8n1E7OsHA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ns-3-users+unsubscribe@googlegroups.com.

[Biljana Bojovic]

Hi Andrei,

yes, there are some default configurations defined. I was thinking e.g on manual configuration of DlSubBandOffset and DlSubBandwidth depending on the positions and measured interference. If you checked and  this configuration is ok in your scenario, then it might be that the trade off between the gained SINR and reduced bandwidth does not pay off. This depends on the link to system mapping curves. So, it can happen that hard ffr is in this scenario limiting the system performance. You should check MCS and TB size for both cases. If any of these suggestions does not help it would be good if you would share the PHY and MAC traces files for both simulation cases.

Cheers,

Biljana

[Andrei Marinescu]

Hi Biljana,

The default configuration for 50 RBs downlink made sense to me, so I left it as such (wrt to DlSubBandwidth). My assumption was that once manual configuration of DlSubBandOffset occurs, this needs to be adjusted accordingly for all neighbouring sectors to avoid interference between each other. If neighbours configurations get adjusted, this creates a chain effect that thus spreads in the whole network and might lead to further issues (e.g., a configuration that is good for one sector might not be good for another one, so a globally benefiting configuration needs to be chosen).

Tradeoff between gained SINR and reduced bandwidth: there should be at least some noticeable effect for edge users, despite the reduced overall bandwidth. While full reuse is more spectrum efficient (by almost a factor of 2 acc. to the CDFs), it should also lead to more interference for edge users. I've used both MiErrorModel (based on the Vienna BLER curves) and PiroEW2010 to see if the difference between the two in terms of SINR to throughput conversion will have any effect, but the difference was not significant (i.e, full reuse is still better than hard reuse in both cases for all edge users).

I have checked the MCS and transport block sizes- see attached for histograms for both full reuse (full_reuse_hist.tif) and hard reuse (hard_reuse_hist.tif) cases. These somewhat makes sense, although it's interesting to notice that the odd numbered MCSs are skipped (possibly to reduce the computational complexity of the simulator?). Note that for these I have used the DlTxPhyStats traces as I did not output Mac traces in my simulations.

The traces files can be found in this google drive folder. The phy and sinr traces are outputs of 5 concurrent simulations, therefore the asynchronous timing in them (limited to just 5 samples/runs as otherwise some files would be massive!), while the Rlc traces include data for all the 30 runs in each reuse case. 

I've also added the CDFs of the simulations considering 1s of real-time instead of 100ms of realtime for each sample as was the case in the initial post (which lead to the stepped CDF curves). I'm wondering if I should increase this even further.

Cheers,
Andrei

[Andrei Marinescu]

I've also attached the scaled versions of the two histograms for easier comparison.

[Zoraze Ali]

Hi Andrie,

Other than setting the frAlgorithmType == "ns3::LteFrHardAlgorithm" you need to configure FrCellTypeId attribute for every eNB. Without setting this parameter the LteFrHardAlgorithm will not be able to divide the bandwidth properly. Please refer to lena-frequency-reuse.cc for more info.

Kind regards,
Zoraze

[Andrei Marinescu]

Hi Zoraze,

Thanks for your suggestion. It turns out I was only setting the FrCellTypeId  in my other scenarios, but forgot to insert it in the original 3GPP validation scenario with frequency reuse. The cell-edge performance I obtain now seems reasonable!

Regards,
Andrei 

--