Separate memory cache for each developer

7 views
Skip to first unread message

Alexander Obuhovich

unread,
Sep 20, 2010, 4:44:12 AM9/20/10
to In-Portal Development
I've came across one issue, when using Memory Caching in 5.1.0 version of In-Portal.

For example:
  • there are 2 developers
  • each developer uses it's own copy of in-portal code
  • both developers connect to the same database
When all cache was stored in database, then 2nd developer will immediately have changed, that 1st developer have cached. For example, when 1st developer adds new menu item on Front-End, then 2nd developer will see it immediately, since it's cached to "cms_menu" cache.

In memory caching implementation we have protection against case, when 2 websites on In-Portal will use common cache instead of separate. This protection has side effect, that 1st and 2nd developers now will have own cache versions and 2nd developer will have to rebuild cache to see, what 1st developer have placed into it (since cache is build based on common database in my example).

This all happens because of each memory cache key has prefix based on In-Portal installation folder full path (FULL_PATH constant), which is different for both developers. If we will replace it with relative In-Portal installation folder (BASE_PATH constant), then it will do us no good, since it will be "/" for most of the sites and as a result will mix up caches of 2 separate In-Portal installations on same server.

I really don't know if this is a bug or a feature, since database is shared, but cached data isn't.


I suddenly got idea, how to fix that:
Here is problematic line $site_key = 'site_serial:' . crc32(FULL_PATH); of kCache::_cachePrefix method. We can do 2 things here:
  1. move it to new class attribute, so we don't have to calculate crc32 for each stored/retrieved key + replace $site_key with that attribute usage;
  2. use BASE_PATH constant instead of FULL_PATH, when we have $_CONFIG['Misc']['Mode] = 'development'; (new config variable) set in config.php file.


--
Best Regards,

http://www.in-portal.com
http://www.alex-time.com

Dmitry Andrejev

unread,
Sep 20, 2010, 11:19:39 AM9/20/10
to in-por...@googlegroups.com
Hi Alex,

This is both bug and feature since moving site serial to a class attribute will make it more efficient while adding new development mode makes it more as feature. I think we can add test and add this to 5.1.1-B1 if not to much trouble to test for you guys.


DA. 


--


Best regards,

Dmitry A.

Alexander Obuhovich

unread,
Sep 20, 2010, 12:21:48 PM9/20/10
to in-por...@googlegroups.com

We could also get desired effect if database connection string + table prefix will be used instead of FULL_PATH, e.g."mysql://user:pass@localhost:inp_". This way we won't need to add anything into config.php .

This way installations, that are using same database on same seever and with same table prefix will use same caching key as well.

On Sep 20, 2010 6:20 PM, "Dmitry Andrejev" <dand...@gmail.com> wrote:
> Hi Alex,
>
> This is both bug and feature since moving site serial to a class attribute
> will make it more efficient while adding new development mode makes it more
> as feature. I think we can add test and add this to 5.1.1-B1 if not to much
> trouble to test for you guys.
>
>
> DA.
>
>
>
>
> On Mon, Sep 20, 2010 at 3:44 AM, Alexander Obuhovich <aik....@gmail.com>wrote:
>
>> I've came across one issue, when using Memory Caching in 5.1.0 version of
>> In-Portal.
>>
>> For example:
>>

>> - there are 2 developers
>> - each developer uses it's own copy of in-portal code
>> - both developers connect to the same database


>>
>> When all cache was stored in database, then 2nd developer will immediately
>> have changed, that 1st developer have cached. For example, when 1st
>> developer adds new menu item on Front-End, then 2nd developer will see it
>> immediately, since it's cached to "cms_menu" cache.
>>
>> In memory caching implementation we have protection against case, when 2
>> websites on In-Portal will use common cache instead of separate. This
>> protection has side effect, that 1st and 2nd developers now will have own
>> cache versions and 2nd developer will have to rebuild cache to see, what 1st
>> developer have placed into it (since cache is build based on common database
>> in my example).
>>
>> This all happens because of each memory cache key has prefix based on
>> In-Portal installation folder full path (FULL_PATH constant), which is
>> different for both developers. If we will replace it with relative In-Portal
>> installation folder (BASE_PATH constant), then it will do us no good, since
>> it will be "/" for most of the sites and as a result will mix up caches of 2
>> separate In-Portal installations on same server.
>>
>> I really don't know if this is a bug or a feature, since database is
>> shared, but cached data isn't.
>>
>>
>> I suddenly got idea, how to fix that:

>> Here is problematic line *$site_key = 'site_serial:' . crc32(FULL_PATH);*of
>> *kCache::_cachePrefix* method. We can do 2 things here:
>>
>> 1. move it to new class attribute, so we don't have to calculate crc32


>> for each stored/retrieved key + replace $site_key with that attribute usage;

>> 2. use BASE_PATH constant instead of FULL_PATH, when we have *$_CONFIG['Misc']['Mode]
>> = 'development';* (new config variable) set in config.php file.

Dmitry Andrejev

unread,
Sep 20, 2010, 12:55:48 PM9/20/10
to in-por...@googlegroups.com
Nope, it won't work in cases when we have 2+ DB servers (different IPs) which is already the case with one of the projects. I wouldn't go this path.

Let's look at this again - we need to have 2 different copies of the site (the same or different servers) working with the same database be able to share the same Serial Key for Memory Caching.

The case you are describing is VERY rare and in most cases would occur in the development. I believe we can and should add NEW "SiteMode" (development or live) variable to the config which will allow us to resolve described issue and in the future refer to this new config value for more for more flexibility on if Website is in Production or Development.

In other words, I say let's go with your initial idea.

DA.

Alexander Obuhovich

unread,
Sep 20, 2010, 2:32:48 PM9/20/10
to in-por...@googlegroups.com

Solution, used right now won't work for your 2 servers too, since each of your servers will have different FULL_PATH :)

At least my 2nd solution won't break anything or require additional knowledge from programmers.

Dmitry Andrejev

unread,
Sep 20, 2010, 2:35:58 PM9/20/10
to in-por...@googlegroups.com
Not sure why you think it won't work - it's the same FULL_PATH and BASE_PATH on these servers. The idea is that they are identical.

DA

Alexander Obuhovich

unread,
Sep 20, 2010, 2:50:04 PM9/20/10
to in-por...@googlegroups.com

I didn't knew that. I'll hope, that developers won'be forgetting to set proper mode in future. I will stick to my original idea then.

Dmitry A.

unread,
Sep 28, 2010, 11:25:15 AM9/28/10
to In-Portal Development Team
After some additional research looks like we can use the DB connection
credentials to build up the KEY Serial as was proposed by Alex.

Alex would you please create a task and describe what needs to be done
there.

Thanks.

DA.

On Sep 20, 1:50 pm, Alexander Obuhovich <aik.b...@gmail.com> wrote:
> I didn't knew that. I'll hope, that developers won'be forgetting to set
> proper mode in future. I will stick to my original idea then.
>
> On Sep 20, 2010 9:36 PM, "Dmitry Andrejev" <dandre...@gmail.com> wrote:
>
>
>
> > Not sure why you think it won't work - it's the same FULL_PATH and
> BASE_PATH
> > on these servers. The idea is that they are identical.
>
> > DA
>
> > On Mon, Sep 20, 2010 at 1:32 PM, Alexander Obuhovich <aik.b...@gmail.com
> >wrote:
>
> >> Solution, used right now won't work for your 2 servers too, since each of
> >> your servers will have different FULL_PATH :)
>
> >> At least my 2nd solution won't break anything or require additional
> >> knowledge from programmers.
>
> >> On Sep 20, 2010 7:56 PM, "Dmitry Andrejev" <dandre...@gmail.com> wrote:
> >> > Nope, it won't work in cases when we have 2+ DB servers (different IPs)
> >> > which is already the case with one of the projects. I wouldn't go this
> >> path.
>
> >> > Let's look at this again - we need to have 2 different copies of the
> site
> >> > (the same or different servers) working with the same database be able
> to
> >> > share the same Serial Key for Memory Caching.
>
> >> > The case you are describing is VERY rare and in most cases would occur
> in
> >> > the development. I believe we can and should add NEW "SiteMode"
> >> (development
> >> > or live) variable to the config which will allow us to resolve
> described
> >> > issue and in the future refer to this new config value for more for
> more
> >> > flexibility on if Website is in Production or Development.
>
> >> > In other words, I say let's go with your initial idea.
>
> >> > DA.
>
> >> > On Mon, Sep 20, 2010 at 11:21 AM, Alexander Obuhovich <
> >> aik.b...@gmail.com>wrote:
>
> >> >> We could also get desired effect if database connection string + table
> >> >> prefix will be used instead of FULL_PATH,
> >> e.g."mysql://user:pass@localhost:inp_".
> >> >> This way we won't need to add anything into config.php .
>
> >> >> This way installations, that are using same database on same seever
> and
> >> >> with same table prefix will use same caching key as well.
>
> >> >> On Sep 20, 2010 6:20 PM, "Dmitry Andrejev" <dandre...@gmail.com>
> wrote:
> >> >> > Hi Alex,
>
> >> >> > This is both bug and feature since moving site serial to a class
> >> >> attribute
> >> >> > will make it more efficient while adding new development mode makes
> it
> >> >> more
> >> >> > as feature. I think we can add test and add this to 5.1.1-B1 if not
> to
> >> >> much
> >> >> > trouble to test for you guys.
>
> >> >> > DA.
>
> >> >> > On Mon, Sep 20, 2010 at 3:44 AM, Alexander Obuhovich <
> >> aik.b...@gmail.com

Dmitry A.

unread,
Oct 4, 2010, 12:07:24 AM10/4/10
to In-Portal Development Team
New task created:


877: Single Cache Serial-key for Developers on the same Server

http://tracker.in-portal.org/view.php?id=877

DA

Dmitry A.

unread,
Mar 31, 2011, 12:08:25 PM3/31/11
to in-por...@googlegroups.com
New issue related to this task/discussion has been found and filed here (posted for the future reference):



DA
Reply all
Reply to author
Forward
0 new messages