Creating MemoryExampleTable
Venkatesh U
I am summarizing my understanding below, please correct me if i am wrong
1. I can use the logic used in the method public ArcTable execute() of the class AConstructSparqlSampler to get a ArcTable
2. Then I can use the logic used in the class ArcTableToMemoryExampleTableConverter to convert it in to a MemoryExampleTable
What is MemoryExampleTable ? I see it is a class related to Rapid miner
I can understand that an Arc is a jenabean which represents the predicate in a statement. ( Hope this is correct)
Can you explain me more on the structure of the ArcTable class ? ( You can assume that I have an understanding of the way imported data is saved in inqle. i.e each subject contains the predicate hasData whose value is the row data )
So to proceed with a rapid miner experiement for clustering, may be i need to write a new sampler which will return an ArcTable and then convert this ArcTable in to a rapid miner model and work on it.Please correct me if am wrong.
--
Thanks and Best Regards,
Venkatesh Umaashankar
Mailto : venka...@gmail.com
Arise, awake and stop not till the goal is reached - Swami Vivekanandha
David Donohue
Let's see.
MemoryExampleTable is a class in RapidMiner API which can be fed into
RapidMiner, for experimentation. Any/every RapidMiner experiment is
fed with data of this format.
I think the existing sampler can suffice for clustering, since it
starts with inputs of datamodel(s), subject, predicates, and it
outputs an ArcTable.
An ArcTable is a table of Arcs. An Arc is a concept for flattening
RDF into a table. An arc consists of multiple steps. Each step
represents a hop from 1 predicate to the next.
Like
Venki hasHomeTown Chennai haspopulation 100000
So inqle can start with row's subject Venki and have a column
hometoen->population, with a value 100000
First create a new bundle/project
File -> new -> Project... -> Plug-In Project
named say
org.inqle.experiment.rapidminer.clustering
It looks like we need a new LearningCycle class. So just as we have
an ExperimenterAgent for doing regression or classification, we
probably need a ClustererAgent for doing clustering. This
ClusteringAgent will repeatedly run a ClusteringCycle. So you might
start by creating ClusteringCycle by copying LearningCycle. Need to
modify other classes.
Add something like this to plugin.xml
<extension
point="org.inqle.experiment.rapidminer.IRapidMinerExperiment">
<rapidminer-experiment
id="org.inqle.experiment.clustering.MyClusteringExperiment"
file="/org/inqle/experiment/rapidminer/clustering/xml/MyClusteringExperiment.xml"
name="My Clustering Experiment"
description="whatever"
type="clustering"
weight="3000">
</rapidminer-experiment>
</extension>
Will need to add other stuff to plugin.xml to declare the new agent
and the new views etc. See plugin.xml in
org.inqle.experiment.rapidminer.
<extension
point="org.inqle.agent.rap.IAgentFactory">
<agent
class="org.inqle.experiment.rapidminer.agent.ClusteringExperimenterAgentFactory"
name="Custering Agent"
description="This agent runs learning cycles, generating
clustering results"
weight="3000"
access="admin"/>
</extension>
David Donohue
So if the goal is to make a RapidMiner rules object instead of
clustering, I would follow the same steps in this thread. Instead of
ClusteringCycle, you could create a RulesCycle or whatever. The
interesting part will be converting the output from a RapidMiner rules
object of some kind into the appropriate structure of RDF.
Consult bundle org.inqle.experiment.rapidminer for direction. You
could even consider cloning this project as a starting point, as I
think almost everything in that bundle has an analog in your bundle.
May I suggest this approach
(1) Make sure your RM experiment outputs an object containing all the
information that might be of use.
(2) Figure out which class in the RM API that this object corresponds
to, and consult the RM Javadoc to figure out how to get relevant info
out of RM
(3) Switching to SWRL, figure out a way to model a SWRL rule as a Java
object. SWRL field values should correspond to either literals of
same datatype, or to other Java objects.
(4) Create code for each of these classes. E.g. top level class is
SwrlRule. E.g. we might have SwrlAntecedant and SwrlConsequent
classes, and SwrlRule contains a field for each.
(5) Add annotations to make all these classes into Jenabeans.
(6) To your RulesCycle, add code which extracts relevant rule info
from the RM class, and creates 1 or more SwrlRule objects.
(7) Persist these to the appropriate datamodel.
This is going to be a great addition to inqle. Let me know any questions!
Best,
Dave
David Donohue
org.inqle.rapidminer.experiment.rules
Change MANIFEST.MF to this
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: INQLE RapidMiner Plug-in
Bundle-SymbolicName: org.inqle.experiment.rapidminer.rules;singleton:=true
Bundle-Version: 1.0.0
Bundle-Vendor: INQLING, LLC
Require-Bundle: org.inqle.rapidminer.experiment,
org.inqle.core,
org.inqle.data.rdf.jenabean,
org.inqle.core.extensions,
org.eclipse.rap.jface,
org.eclipse.rap.ui,
org.inqle.ui.rap,
org.inqle.agent,
org.inqle.data.sampling,
org.eclipse.core.databinding,
org.eclipse.core.databinding.beans,
org.eclipse.rap.jface.databinding,
org.eclipse.rap.ui
Import-Package: com.hp.hpl.jena.query,
com.hp.hpl.jena.rdf.model,
com.hp.hpl.jena.util,
junit.framework,
org.eclipse.core.commands.common,
org.eclipse.core.databinding.conversion,
org.eclipse.core.runtime,
org.eclipse.jface,
org.eclipse.jface.action,
org.eclipse.jface.resource,
org.eclipse.jface.util,
org.eclipse.jface.wizard,
org.eclipse.rwt.lifecycle,
org.eclipse.swt,
org.eclipse.swt.graphics,
org.eclipse.swt.widgets,
org.junit,
org.junit.runner,
org.junit.runners,
org.osgi.framework;version="1.4.0",
org.osgi.service.component;version="1.0.0",
org.xml.sax
Bundle-ClassPath: bin/main/,
.
Eclipse-RegisterBuddy: org.inqle.core.extensions,
org.inqle.data.rdf.jenabean,
org.inqle.ui.rap
Bundle-RegisterBuddy: org.inqle.core.extensions,
org.inqle.data.rdf.jenabean,
org.inqle.ui.rap
Eclipse-LazyStart: true
David Donohue
<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.2"?>
<plugin>
<extension
point="org.inqle.data.datamodels">
<datamodel
id="org.inqle.datamodels.rules"
weight="8500"
description="contains experiment results"
cacheInMemory="false"
targetDatabase="org.inqle.data.databases.cache">
</datamodel>
</extension>
<extension
point="org.inqle.experiment.rapidminer.IRapidMinerExperiment">
<rapidminer-experiment
id="org.inqle.experiment.rules.VenkisRuleExperiment"
file="/org/inqle/experiment/rapidminer/rules/xml/VenkisRuleExperiment.xml"
name="Venkis Rules Experiment"
description="whatever"
type="rules"
weight="1000">
</rapidminer-experiment>
</extension>
<extension
point="org.inqle.agent.rap.IAgentFactory">
<agent
class="org.inqle.experiment.rapidminer.agent.ExperimenterAgentFactory"
name="Rules Experimenter Agent"
description="This agent runs rule learning cycles, generating rules"
weight="2000"
access="admin"/>
</extension>
<extension
point="org.eclipse.ui.views">
<view
name="Rules"
icon="icons/rule.jpeg"
class="org.inqle.experiment.rapidminer.rules.RulesView"
id="org.inqle.experiment.rapidminer.rules.rulesView">
</view>
</extension>
</plugin>
David Donohue
<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.2"?>
<plugin>
<extension
point="org.inqle.data.datamodels">
<!--Create a system datamodel, to contain rules. Add this annotation
to your SwrlRule class to make it be persisted here by default:
@TargetDatamodel("org.inqle.datamodels.rules")
-->
<datamodel
id="org.inqle.datamodels.rules"
weight="8500"
description="contains experiment results"
cacheInMemory="false"
targetDatabase="org.inqle.data.databases.cache">
</datamodel>
</extension>
<extension
point="org.inqle.experiment.rapidminer.IRapidMinerExperiment">
<!--Add 1 entry per rapidminer experiment you add. Your Java
code will have to look for rapidminer experiment extensions with
type="rules"
-->
<rapidminer-experiment
id="org.inqle.experiment.rules.VenkisRuleExperiment"
file="/org/inqle/experiment/rapidminer/rules/xml/VenkisRuleExperiment.xml"
name="Venkis Rules Experiment"
description="whatever"
type="rules"
weight="1000">
</rapidminer-experiment>
</extension>
<!--I made a change below. You need your own factory for creating agent-->
<extension
point="org.inqle.agent.rap.IAgentFactory">
<agent
class="org.inqle.experiment.rapidminer.rules.agent.RuleExperimenterAgentFactory"
name="Rules Experimenter Agent"
description="This agent runs rule learning cycles, generating rules"
weight="2000"
access="admin"/>
</extension>
<!--Can add this last. A custom viewer of rules-->
David Donohue
LearningCycle becomes RuleLearningCycle
Alter these methods
public ExperimentResult execute()
private ExperimentResult runDataThroughExperiment(IDataTable
dataTable, IRapidMinerExperiment rapidMinerExperiment) {
to return a SwrlRule (or whatever class is appropriate)
Change
List<IRapidMinerExperiment> acceptableExperiments =
RapidMinerExperimentLister.listMatchingExperiments(dataTable);
to this
List<IRapidMinerExperiment> acceptableExperiments =
RulesExperimentLister.listRulesExperiments();
If you need to select which rules experiment based on info in the data
table you could have complexity as is in the
RapidMinerExperimentLister class. Otherwise, simplify this class to
just return experiments of type "rules"
David Donohue
SwrlRule, SwrlAntecedant, etc.
Next the classes
ExperimenterAgent,
ExperimenterAgentFactory
ExperimenterAgentWizard
Change these to
RulesExperimenterAgent,
RulesExperimenterAgentFactory
RulesExperimenterAgentWizard
Need to modify some behavior probably. At a minimum, change
RulesExperimenterAgent to deal with
IRuleLearningCycle,
RuleLearningCycle
[Not sure the merits of having the interface it it failed this current
use case of adding a new RM experimenter type!]