Redis on Windows - Dušan Majkić - is good enough just for PUB/SUB?

[Carlos Mendes]

Hi,

we’re considering using Redis as a distributed cache.

Our applications run on the .NET stack and 95% of our customers don’t really need a distributed cache to support their workload.

For the 5% that really need it is OK for us to go through the extra work of setting up linux servers.

In any case, we would like to use Redis Pub/Sub mechanism in every customer (to invalidate local caches).

Could someone briefly summarize why the build from Dušan Majkić is not production ready for Windows?

Is it good enough for PUB/SUB?

Thanks in advance 

[Josiah Carlson]

On Sat, Feb 25, 2012 at 4:51 PM, Carlos Mendes
<carlos.a...@gmail.com> wrote:
> Hi,
>
> we’re considering using Redis as a distributed cache.
>
> Our applications run on the .NET stack and 95% of our customers don’t really
> need a distributed cache to support their workload.
>
> For the 5% that really need it is OK for us to go through the extra work of
> setting up linux servers.

Then set up Linux servers ;)

> In any case, we would like to use Redis Pub/Sub mechanism in every customer
> (to invalidate local caches).
>
> Could someone briefly summarize why the build from Dušan Majkić is not
> production ready for Windows?

Background saving and AOF rewriting (BGSAVE, BGREWRITEAOF, as well as
the BGSAVE that is induced when a slave connects to a master) is
implemented in Linux as a fork operation. Since fork doesn't exist on
Windows, those commands have been replaced with ones that don't fork,
and will block the server until the save/rewrite is complete. For most
people who want to save their data, this makes the Windows build
unusable. There have been discussions on how to remove the need to
fork, but none have been implemented.

> Is it good enough for PUB/SUB?

Yes.

Regards,
- Josiah

[Carlos Mendes]

Thank you Josiah.

Being able to use the same mechanism for PUB/SUB everywhere and just dealing with the distributed caching when needed (using Linux) is great news!

[Salvatore Sanfilippo]

Hi Carlos, I want to add some fine print here ;)

The win32 ports of Redis are not "official" and we'll not be able to
help if it crashes or if there are other issues.
From our point of view those ports are cool projects, but we don't
support them because it's a modified code base.

Cheers,
Salvatore

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/P9X7fzx1wHcJ.
>
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware

http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele

[Carlos Mendes]

Thank you for the heads-up Salvatore.

[Salvatore Sanfilippo]

On Sun, Feb 26, 2012 at 8:51 PM, Carlos Mendes
<carlos.a...@gmail.com> wrote:

> Thank you for the heads-up Salvatore.

You are welcome, I wonder if you are also aware that there is a new
win32 port made by Windows people, partially based on the code (or
ideas?) written by Dušan. This port used to be a patch, but now there
is a repository and so forth.

https://github.com/Microsoft-Interop/redis

Cheers,
Salvatore

[Carlos Mendes]

Yes, I know. But I would prefer to use the official builds, at least for using Redis as a distributed cache.

Personally I don't have any problems with Linux. The problem is that some customers have really small technical teams and are kind of "afraid" of the unknown (Linux).

But I think we can address this by just deploying a VM... Kind of Linux (and Redis)-as-a-"black-box".

Thks again

[Claudio]

Just chiming in to provide some details.

Thanks Salvatore for pointing out our effort to port Redis on Windows (that is based on both Dusan's code and ideas).

The current version of the code is still a prototype and definitively not ready for 'production' use. We are about 2 weeks away from releasing a new version of our code that should provide a better solution for the Fork/COW mechanism that is not available on Windows. If we don't see a big gap in performance between our implementation and Linux we will focus on testing and provide a stable version that could be considered 'beta', not yet ready for production but definitively more stable. In the meanwhile we will focus on testing another solution and see if it get us even better performance, anyway our roadmap is available (https://github.com/Microsoft-Interop/redis/wiki/Ideas-for-snapshotting-improvements-and-current-roadmap-for-Redis-on-Windows).

Claudio

https://github.com/Microsoft-Interop/redis

Cheers,
Salvatore

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.

[Luis Lavena]

Hello Claudio,

On Mon, Feb 27, 2012 at 2:37 PM, Claudio <ccal...@hotmail.com> wrote:
> Just chiming in to provide some details.
>

> [snip]

>
> If we don't see a big gap in performance between our implementation and Linux we will focus on testing and provide a stable version that could be considered 'beta', not yet ready for production but definitively more stable. In the meanwhile we will focus on testing another solution and see if it get us even better performance, anyway our roadmap is available (https://github.com/Microsoft-Interop/redis/wiki/Ideas-for-snapshotting-improvements-and-current-roadmap-for-Redis-on-Windows).

Thanks for sharing the roadmap and the details.

Something I noticed in Microsoft-Interop fork compared to Dusan is the
dependency and usage of pthreads-win32

Is there any plan to move to a shim approach where uses Windows API
more directly?

Thank you.
--
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

[Salvatore Sanfilippo]

On Mon, Feb 27, 2012 at 6:37 PM, Claudio <ccal...@hotmail.com> wrote:

> The current version of the code is still a prototype and definitively not ready for 'production' use. We are about 2 weeks away from releasing a new version of our code that should provide a better solution for the Fork/COW mechanism that is not available on Windows

Hello Claudio,

thank you for sharing. About the Fork/COW mechanism, I think there are
ways to do it so that will perform pretty well at least for small
values. Actually for small values that are specially encoded in
ziplists/zipmaps/... it is possible to do it *better* than OS copy on
write, because to duplicate the value all you have to do is
alloc+memcpy of a piece that is smaller than a 4096 page.

Probably for bigger objects it will be slower, like big lists, but at
least the implementation has some interesting point where it is also
better, and it's pretty cool.

[Claudio]

It is in the list of things we are considering but it is not super-critical
right now so it will have to wait a little longer :)

Right now our main goal is to find a solution for the Fork/COW problem and
find a good balance between the benefit of each solution and the
intrusiveness into the Redis code base, we don't want to end up with having
to spend a huge amount of time to integrate new releases of the main code
base. I think this effort makes sense only if we can keep the Windows
implementation in-line with the latest stable version available on Linux so
we need to be able to integrate new releases quickly.

Claudio

-----Original Message-----
From: redi...@googlegroups.com [mailto:redi...@googlegroups.com] On Behalf
Of Luis Lavena
Sent: Monday, February 27, 2012 11:10 AM
To: redi...@googlegroups.com
Subject: Re: Redis on Windows - Dušan Majkić - is good enough just for
PUB/SUB?

Hello Claudio,

--

[Claudio]

This is in line with what we expect and we are focusing our tests on big
objects so we have some numbers about the worst case scenario. Not sure what
else we will be able to do about that but at least we have a solution that,
as you said, offers some benefits. After that we will look into using the
MEM_WRITE_WATCH flag in Windows so leveraging more the OS features to mimic
the Fork/COW behavior.

But I also hope to get some feedback from You and the Redis community to
help us to focus on the right scenarios.

Claudio

-----Original Message-----
From: redi...@googlegroups.com [mailto:redi...@googlegroups.com] On Behalf
Of Salvatore Sanfilippo
Sent: Monday, February 27, 2012 11:16 AM
To: redi...@googlegroups.com
Subject: Re: Redis on Windows - Dušan Majkić - is good enough just for
PUB/SUB?

Hello Claudio,

Cheers,
Salvatore

--