I was working in Cascading a month back. Now we are trying to implement the same in Scalding. I have one basic question.
How can i define my source & sink schema in Scalding ??
Below is the procedure that we followed in Cascading
SrcFields sourcefields = new SrcFields();
SinkFields sinkfields = new SinkFields();
Fields source = sourcefields.sourceFields();
Fields sink = sinkfields.sinkfields();
Scheme sourceScheme = new TextDelimited(source,",");
Scheme sinkScheme = new TextDelimited(sink,",");
In Scalding, you can use either Fields based or Typed interface, as per Source documentation. In former, you would use Csv or Tsv classes to read or write.
For typed interface, you would use TypedCsv or TypedTsv classes.
You can find examples in Scalding Tutorial: https://github.com/twitter/scalding/blob/develop/tutorial/Tutorial6.scala , https://github.com/twitter/scalding/blob/develop/tutorial/TypedTutorial.scala
Related
I am going to register interests in ALL_KEYS for my Pivotal GemFire client via Spring Data GemFire, but I find that ClientRegionFactoryBean has one method.
org.springframework.data.gemfire.client.ClientRegionFactoryBean.setInterests(Interest<MyRegionPojo>[] interests)
In this case, I only can set the exact keys, but I want to register interests for all keys. My key is not a simple class like String, or Long, but a complex object MyRegionPojo.
Please help if any method to implement so like GemFire API region.registerInterest("ALL_KEYS");
You problem statement is a bit vague but I assume/suspect you are configuring your Spring (Data GemFire) (SDG) application using Spring JavaConfig?
However, I will quickly add that this is not unlike how you would register interests in all keys using SDG's XML namespace, as shown here.
The JavaConfig approach is similar, but clearly based on "strongly-typed arguments", namely 1 or more sub-type instances of the o.s.d.g.client.Interest class to the o.s.d.g.client.ClientRegionFactoryBean.setInterests(:Interest<K>[]) method.
By way of example, you might do the following...
#Bean("Example")
public ClientRegionFactoryBean<?, ?> exampleRegion(GemFireCache gemfireCache) {
ClientRegionFactoryBean<MyRegionKey, MyRegionValue> exampleRegion =
new ClientRegionFactoryBean<>();
RegexInterest regexInterest = new RegexInterest();
regexInterest.setKey(".*");
exampleRegion.setCache(gemfireCache);
exampleRegion.setShortcut(ClientRegionShortcut.PROXY);
exampleRegion.setInterests(new Interest[] { regexInterest });
exampleRegion.setKeyConstraint(MyRegionKey.class);
exampleRegion.setValueConstraint(MyRegionValue.class);
return exampleRegion;
}
NOTE: updated the example above to reflect the proper way to register (Regex) interests based on SDG 1.9 or earlier. Keep in mind that the `o.s.d.g.client.RegexInterest.getRegex() delegates to getKey() therefore you can set the Regular Expression using setKey(:String) as I have shown above.
Notice the o.s.d.g.client.RegexInterest sub-type registration, which is effectively the same as register interests in "ALL_KEYS", as described here as well.
Hope this helps!
-John
In Gio.Settings I can list relocatable schemas using
Gio.Settings.list_relocatable_schemas()
and I can use
Gio.Settings.new_with_path(schema_id, path)
to get a Gio.Settings instance. But how can I get all value for path that are currently used for a given schema_id?
Normally, a schema has as fixed path that determines where the
settings are stored in the conceptual global tree of settings.
However, schemas can also be ‘relocatable’, i.e. not equipped with a
fixed path. This is useful e.g. when the schema describes an
‘account’, and you want to be able to store a arbitrary number of
accounts.
Isn't the new_with_path just for that? You have to store the schemas somewhere associated with accounts, but that is not the responsibility of the Settings system. I think new_with_path is for the case where your schemas depend on accounts.
I think you can find more information with GSettingsSchemas - this is an example in the Description for a case where the Schema is part of a plugin.
Unfortunately you cannot do it from Gio.Settings.
I see two options here:
Keep separate gsetting to store paths of relocatable schemas
Utilize dconf API, which is a low-level configuration system. Since there is no Python binding (guessing it's Python question) I suggest using ctypes for binding with C.
If you know the root path of your relocatable schemas you can use below snippet list them.
import ctypes
from ctypes import Structure, POINTER, byref, c_char_p, c_int, util
from typing import List
class DconfClient:
def __init__(self):
self.__dconf_client = _DCONF_LIB.dconf_client_new()
def list(self, directory: str) -> List[str]:
length_c = c_int()
directory_p = c_char_p(directory.encode())
result_list_c = _DCONF_LIB.dconf_client_list(self.__dconf_client, directory_p, byref(length_c))
result_list = self.__decode_list(result_list_c, length_c.value)
return result_list
def __decode_list(self, list_to_decode_c, length):
new_list = []
for i in range(length):
# convert to str and remove slash at the end
decoded_str = list_to_decode_c[i].decode().rstrip("/")
new_list.append(decoded_str)
return new_list
class _DConfClient(Structure):
_fields_ = []
_DCONF_LIB = ctypes.CDLL(util.find_library("dconf"))
_DCONF_LIB.dconf_client_new.argtypes = []
_DCONF_LIB.dconf_client_new.restype = POINTER(_DConfClient)
_DCONF_LIB.dconf_client_new.argtypes = []
_DCONF_LIB.dconf_client_list.argtypes = [POINTER(_DConfClient), c_char_p, POINTER(c_int)]
_DCONF_LIB.dconf_client_list.restype = POINTER(c_char_p)
You can't, at least not for an arbitrary schema, and this is by definition of what a relocatable schema is: a schema that can have multiple instances, stored in multiple arbitrary paths.
Since a relocatable schema instance can be stored basically anywhere inside DConf, gsettings has no way to list their paths, it does not keep track of instances. And dconf can't help you either, as it has no notion of schemas at all, it only knows about paths and keys. It can list the subpaths of a given path, but that's about it.
It's up for the application, when creating multiple instances of a given relocatable schema, to store each instance in a sensible, easily discoverable path, such as a subpath of the (non-relocatable) application schema. Or store the instance paths (or suffixes) as a list key in such schema.
Or both, like Gnome Terminal does with its profiles:
org.gnome.Terminal.ProfilesList is a non-relocatable, regular schema, stored at DConf path /org/gnome/terminal/legacy/profiles:/
That schema has 2 keys, a default string with a single UUID, and a list list of strings containing UUIDs.
Each profile is an instance of the relocatable schema org.gnome.Terminal.Legacy.Profile, and stored at, you guess... /org/gnome/terminal/legacy/profiles:/:<UUID>/!
This way a client can access all instances using either gsettings, reading list and building the paths from the UUIDs, or from dconf, by directly listing the subpaths of /org/gnome/terminal/legacy/profiles:/.
And, of course, for non-relocatable schemas you can always get their paths with:
gsettings list-schemas --print-paths
EF People,
My understanding is that the newly made public APIs for metadata will allow us to add enough metadata in to the model so that TVF can be called and be composable.
If anyone can point me in the right direction I would greatly appreciate it. Without Composable TVF I have to jump through some major work a rounds.
From looking at the unit test it looks like something a long this line of thought:
var functionImport = EdmFunction.Create()
"Foo", "Bar", DataSpace.CSpace,
new EdmFunctionPayload
{
IsComposable = true,
IsFunctionImport = true,
ReturnParameters = new[]
{
FunctionParameter.Create("functionname", EdmType.GetBuiltInType()
EdmConstants.ReturnType,
TypeUsage.Create(collectionTypeMock.Object),
ParameterMode.ReturnValue),
}
});
...
entityContainer.AddFunctionImport(functionImport);
Thanks,
Brian F
Yes, it is now possible in EF6.1. I actually created a custom model convention which allows using store functions in CodeFirst using the newly opened mapping API. The convention is available on NuGet http://www.nuget.org/packages/EntityFramework.CodeFirstStoreFunctions. Here is the link to the blogpost containing all the details: http://blog.3d-logic.com/2014/04/09/support-for-store-functions-tvfs-and-stored-procs-in-entity-framework-6-1/. The project is open source and you can get sources here: https://codefirstfunctions.codeplex.com/
I want to create a very simple query to look up a sqlite db using greendao. 2 fields, one is the ID and the other 'affirmation'.
i am sorry to be such a beginner, but i am not sure how to use greendao including what to import etc.. All i have been able to do so far is add the greendao libraries but i cant find a good tutorial to just do a query. Basically i want it to be a random ID that calls up a random affirmation and return it to my main activity.. Once again i am sorry but i am really trying and getting nowhere..
Greendao is a ORM-framework. If you don't know what this means you should look up this first.
Greendao generally works as follows:
You create a java-project that generates your sourcecode for your real app. You have to include DaoCore and DaoGenerator in this project.
You add the generated sourcecode to your android-project and include DaoCore in it. DaoGemerator is not neccessary.
For examples how to generate the code and define your entities the greendao-website is a good place to go.
According to your description you need an entity with id-property and a string-property (affirmation).
In your android-project you then use the DevOpenHelper to get a session and from the session you can get the dao (Data Access Object) for your entity. The dao includes the very basic query to load data by id (load ()).
Please notice that the DevOpenHelper is only meant for development process. For your final release you should extend OpenHelper and costumize your actions to be taken on DB-schema update.
Here is some example code I have in my application.
DaoHelper.getInstance().getDaoSession().clear();
OperationDao dao = DaoHelper.getInstance().getDaoSession().getOperationDao();
String userId = "some id"
WhereCondition wc1 = new WhereCondition.PropertyCondition(OperationDao.Properties.UserId,
" = " + userId);
WhereCondition wc2 = new WhereCondition.PropertyCondition(OperationDao.Properties.Priority,
" > " + 4);
// Uncached is important if your data may have changed recently.
List<Operation> answer = dao.queryBuilder().where(wc1, wc2).listLazyUncached();
This is a decent tutorial on how to learn greendao. Make sure you follow the links to the further parts.
You can use:
daoMaster = new DaoMaster(db);
daoSession = daoMaster.newSession();
yourDao = daoSession.getYourDao();
Random() random = new Random();
List<YourObject> objects = yourDao.loadAll();
YourObject yourObject = objects.get(random.nextInt(objects.size());
I have the following in a config.ini file: (Zend_Form_Element)
site_status.name = "site_status"
site_status.type = "select"
site_status.label = "Status"
site_status.options.multiOptions.active.key = "Active"
site_status.options.multiOptions.active.value = "Active"
site_status.options.multiOptions.active.key = "Inactive"
site_status.options.multiOptions.active.value = "Inactive"
As you can see this is supposed to be a dropdown (select) box, however it is being rendered as a standard text box. What am I doing wrong?
--> Edit
Rather than tying the elements to a form, I am trying to tie them to a database: In my code it would look something like this:
[{tablename}] // the table name would represent a section in the ini
{column}.name = "{column_name/form_field_id}";
{column}.type = "{form_element_type}"
{column}.label = "{form_element_label}"
...
From there I would pull in the database table(s) that the form would represent data for (one or more tables as necessary). As far as the reasoning for this approach is that (down the road), I want to define (either by ini or some other storage method), a configuration file that would be a list of fields/elements that belong to a specific form (that a non-programmer type could easily edit), that the 'generic' form class would read, pull in the element info, and create the form on the fly.
I do realize however this poses another problem which I haven't yet figured out, and that is how to use table lookups for select elements (without coding the database retrieval of the lookup into the form, so that a non-user could easily just define it without any programming, purely configuration, but that is a whole other topic not part of my question here. (and I think I have viable ideas/solutions to that part of the problem anyhow) -- extra config entries and a generic routine pretty much.
I hope that clarifies my thought process and reason why I am doing it the way I am in the example above.
I have not yet played with using a Zend_Config to construct an instance of Zend_Form.
But a look at the code suggests that Zend_Form::addElement() doesn't directly take a Zend_Config instance as a param. Rather, it looks like you need pass your Zend_Config instance to the form constructor. It also seems that the config format needs to be a little deeper in order to map config keys to setXXX() calls.
In path/to/config/myForm.ini:
[myForm]
myForm.elements.site_status.name = "site_status"
myForm.elements.site_status.type = "select"
myForm.elements.site_status.label = "Status"
myForm.elements.site_status.options.multiOptions.active.key = "Active"
myForm.elements.site_status.options.multiOptions.active.value = "Active"
myForm.elements.site_status.options.multiOptions.inactive.key = "Inactive"
myForm.elements.site_status.options.multiOptions.inactive.value = "Inactive"
Then instantiating:
$formConfig = new Zend_Config_Ini('path/to/config/myForm.ini', 'myForm');
$form = new Zend_Form($formConfig);
Not tested, but looking at this example:
Using Zend_Form with Zend_Config - Andrew Vayanis
it feels like it should go something like the above.
Update
In view of the comments/feedback from #Aaron, two more approaches.
We could extend Zend_Form, implementing a method called something like addElementByConfig in which we would pass the shallow Zend_Config instance that describes the element itself. In fact, we could even just override addElement(), taking a recursive approach: if the first param is an instance of Zend_Config, then call addElement() using the component data.
If the atomicity and re-usability are the primary benefits we seek in using Zend_Config to describe an element, then perhaps we just make a custom element extending Zend_Form_Element. Then we could use these elements in any forms we wish.