MacOS gui+cli

[Alex Leverington]

All,

I've written the initial implementation of a nosql GUI for MacOS -- with initial support for Redis. My long-term goal is to adapt this for use with several key/value/hash storage engines. Initial features will be for console access, monitoring, and visualizing results. The first release is in-review w/apple and it has basic cli support.

I'm a little overwhelmed with the number of features I *could* build but would love to hear from others. If you're interested in a specific feature for the initial release please feel free to email me (on or off list).



Thanks,
Alex Leverington

[Dvir Volk]

WAT? Apple approves Mac apps too? 

On Sat, Jun 23, 2012 at 10:35 PM, Laks <lakshmi...@gmail.com> wrote:

Hi

Has it been approved with Apple yet? Can't wait to check it out.

Thanks

Laks

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.

To view this discussion on the web visit https://groups.google.com/d/msg/redis-db/-/vGFL3m1I6IcJ.

To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

[Sergei Tulentsev]

for the mac app store - yes.

--
Best regards,
Sergei Tulentsev

[Alex Leverington]

It went into review again Friday so nothing until next week. First review always takes forever.

--

Alex Leverington

On Jun 23, 2012, at 2:35 PM, Laks <lakshmi...@gmail.com> wrote:

Hi

Has it been approved with Apple yet? Can't wait to check it out.

Thanks

Laks

On Tuesday, June 12, 2012 4:45:29 PM UTC-5, Alex Leverington wrote:

--

[Dvir Volk]

I hope you can at least install unlicensed software there.

[Alex Leverington]

It's been approved now. Very basic, now the hard part starts :)

http://itunes.apple.com/us/app/pair/id525617748?ls=1&mt=12

Let me know if you'd like a promo code, I would be happy to give in exchange for good feedback.

--

Alex Leverington

On Jun 23, 2012, at 2:35 PM, Laks <lakshmi...@gmail.com> wrote:

Hi

Has it been approved with Apple yet? Can't wait to check it out.

Thanks

Laks

On Tuesday, June 12, 2012 4:45:29 PM UTC-5, Alex Leverington wrote:

--

[Sergei Tulentsev]

$3.99 for a guified terminal - isn't it too much? :)

[Daniel Schnell]

Hi,

in my application I want to use redis for counting named entities (i.e. words with names) per time range.

Let's say I have many docs flying in over the daytime and extract specific relevant named entities from those. 

1.) Each selected NE should be inserted into redis and counted up per occurence

2.) It should be possible to easily fetch the top counted N named entities per time range. I.e. hourly, daily, weekly, monthly, yearly and also freely selectable time ranges, e.g. 12.11.11 -27.06.12

3.) It should be possible to enter NE's in a "time slot" long after that time slot has passed: the time slot has to be independent from insertion time

4.) Minimal time slot is 1 hour. After two days these will be accumulated in a daily overall count 

I came across this project: https://github.com/antirez/redis-timeseries but Salvatore flags this explicitely as non production ready and furthermore it doesn't allow insertions in the middle, violating criteria 3.)

I could use sorted sets as time slots with NE as key and use ZUNIONSTORE to collect all NE over the given sets. 

I am unsure about how to organize and name these sets to efficiently find the relevant sets for feeding ZUNIONSTORE.

One idea I came up with is to name a set according to the time since the epoch of the day/week/month/year: to select e.g. all daily snapshots of a time range, I could select all "snapshot-daily-ABCDEFG" sorted sets where the bounds of the time range will be converted to since-the-epoch-times and be numerically compared to the -ABCDEFG part of the sorted sets names.

Is this the most efficient way to organize this via Redis data structures ? Is there an upper bound of arguments that can be fed to the ZUNIONSTORE command ?

Regards,

Daniel.

[Josiah Carlson]

Efficiency in querying? Or efficiency in data storage?

Either way, the difference is relatively minimal. My suggestion is to
explicitly keep hourly, daily, weekly, and monthly ZSETs that are kept
updated in real time. You can set expirations of 48 (or 72) hours on
your hourly ZSETs. Why? Because with a pipelined request, adding to 4
ZSETs is not significantly slower than adding to 1 ZSET. The only
thing you need to be careful about is not writing to your hourly ZSETs
when they are more than 48 hours old.

The only question you really have to answer is, do you mean "calendar
week" and "calendar month", or do you mean "weeks since epoch" and
"30-day months since epoch", because that will determine how
pre-aggregate your data. For the weeks since epoch and 30-day months
since epoch, I've done the following quite a few times (in Python):

AGGS = [
('month', 30*86400),
('week', 7*86400),
('day', 86400),
('hour', 3600),
]
CUTOFF = 2*86400

def update_aggregates(conn, entities, timestamp):
when = []
for label, duration in AGGS:
when.append("%s:%s"%(label, int(timestamp / duration)))

if time.time() - timestamp > CUTOFF:
when.pop()

pipe = conn.pipeline(False)
for agg in when:
for entity in entities:
pipe.zincrby(agg, entity, 1)
if len(when) == 4:
pipe.expire(when[-1], CUTOFF)
pipe.execute()

> Is there an upper bound of arguments that can be fed to the ZUNIONSTORE
> command ?

I've not had any issues feeding a few hundred ZSETs into the
ZUNIONSTORE command. That said, I wouldn't use it in this scenario
unless absolutely necessary. Depending on how many named entities you
have (1k? 10k? 100k?), you could find yourself waiting for a while for
Redis to calculate the result while your writing of named entities
gets backed up. You will want to do some testing, and perhaps write to
the master with queries against a slave (remember to turn off
read-only mode on the slaves if you are using Redis >= 2.6).

Regards,
- Josiah

[Daniel Schnell]

Josiah, thanks for your quick reply.

Am 26.06.2012 um 17:19 schrieb Josiah Carlson:


<snip>

>
> Efficiency in querying? Or efficiency in data storage?
>

Efficiency in querying. Data storage size will not be an issue. Also insertions are not more than a couple of thousands per hour.

> Either way, the difference is relatively minimal. My suggestion is to
> explicitly keep hourly, daily, weekly, and monthly ZSETs that are kept
> updated in real time. You can set expirations of 48 (or 72) hours on
> your hourly ZSETs. Why? Because with a pipelined request, adding to 4
> ZSETs is not significantly slower than adding to 1 ZSET. The only
> thing you need to be careful about is not writing to your hourly ZSETs
> when they are more than 48 hours old.
>
> The only question you really have to answer is, do you mean "calendar
> week" and "calendar month", or do you mean "weeks since epoch" and
> "30-day months since epoch", because that will determine how
> pre-aggregate your data.

Yes. I will probably go with the same approach you took: "-time-since-epoch".
That is a fine piece of code you are sharing. Thanks.
If I have adapted it to Ruby, I will post it here as well.

The overall number of different NE's is well over 100k. But for most queries it would be much less, say < 1k, e.g. if the last 48 hours are concerned. I would therefore accumulate the stats of older time ranges on a cron basis and store the results in some cache. I was concerned about redis or the ruby client choking when I feed too many arguments e.g. when accumulating over big time ranges within the cron job.


Regards,

Daniel.

[Bob Hutchison]

On 2012-06-26, at 9:20 AM, Sergei Tulentsev wrote:

$3.99 for a guified terminal - isn't it too much? :)

:-) Well, from what I've heard, getting an app into the AppStore can be quite a hurdle, so it's arguable that that's a good first feature :-)

Even so, I've purchased it... $3.99 isn't too much to see where this goes.

Cheers,

Bob

[Dvir Volk]

not that I'm a mac user, but what does it give me versus opening a console and running redis-cli?

[Alex Leverington]

Window title is updated with memory usage and key count, and you can paste content with newlines.

The app store doesn't allow charging for updates so they're always free which means future features via updates. Lua, better monitoring, tabbed-sessions and a few other things you *can* get with a console but not as seamlessly.

Before those features I must to do more work under the hood to support multiple connections. Some commands such as monitor and pubsub need an option to have their own connection.

--

Alex