Say I have an existing dataset in which something is an embedded document but we want to turn it into a reference instead. Is there any kind of automatic or semi-automatic way to do this refactoring.
Refactoring itself is simple. Just swap "embeds_one" for "has_one" (or substitute appropriate terms for your mapper library). It's the data migration that will cause some pain. Or maybe it won't. Here's a small ruby script I whipped up in under 10 minutes. It should cover what I think you need.
source_collection = 'users'
field_to_expand = 'address'
parent_field = 'user_id'
expanded_collection = 'addresses'
require 'mongo'
db = Mongo::Connection.new.db('test')
users = db.collection(source_collection)
addresses = db.collection(expanded_collection)
# prepare test data
users.remove()
addresses.remove()
users.insert({name: 'Joe', address: {city: 'Rio de Janeiro'}})
users.find().to_a # => [{"_id"=>BSON::ObjectId('50614e910ed4c08a6a000001'), "name"=>"Joe", "address"=>{"city"=>"Rio de Janeiro"}}]
users.find().each do |u|
# move subdocument to a separate collection
addr = u[field_to_expand]
addr[parent_field] = u['_id']
addresses.insert(addr)
# erase from original document
users.update({_id: u['_id']}, {'$unset' => {field_to_expand => 1}})
end
users.find().to_a # => [{"_id"=>BSON::ObjectId('50614e910ed4c08a6a000001'), "name"=>"Joe"}]
addresses.find().to_a # => [{"_id"=>BSON::ObjectId('50614e910ed4c08a6a000002'), "city"=>"Rio de Janeiro", "user_id"=>BSON::ObjectId('50614e910ed4c08a6a000001')}]
Related
I have a perl code like this:
use constant OPERATING_MODE_MAIN_ADMIN => 'super_admin';
use constant OPERATING_MODE_ADMIN => 'admin';
use constant OPERATING_MODE_USER => 'user';
sub system_details
{
return {
operating_modes => {
values => [OPERATING_MODE_MAIN_ADMIN, OPERATING_MODE_ADMIN, OPERATING_MODE_USER],
help => {
'super_admin' => 'The system displays the settings for super admin',
'admin' => 'The system displays settings for normal admin',
'user' => 'No settings are displayed. Only user level pages.'
}
},
log_level => {
values => [qw(FATAL ERROR WARN INFO DEBUG TRACE)],
help => "http://search.cpan.org/~mschilli/Log-Log4perl-1.49/lib/Log/Log4perl.pm#Log_Levels"
},
};
}
How will I access the "value" fields and "help" fields of each key from another subroutine? Suppose I want the values of operating_mode alone or log_level alone?
The system_details() returns a hashref, which has two keys with values being hashrefs. So you can dereference the sub's return and assign into a hash, and then extract what you need
my %sys = %{ system_details() };
my #loglevel_vals = #{ $sys{log_level}->{values} };
my $help_msg = $sys{log_level}->{help};
The #loglevel_vals array contains FATAL, ERROR etc, while $help_msg has the message string.
This makes an extra copy of a hash while one can work with a reference, as in doimen's answer
my $sys = system_details();
my #loglevel_vals = #{ $sys->{log_level}->{values} };
But as the purpose is to interrogate the data in another sub it also makes sense to work with a local copy, what is generally safer (against accidentally changing data in the caller).
There are modules that help with deciphering complex data structures, by displaying them. This helps devising ways to work with data. Often quoted is Data::Dumper, which also does more than show data. Some of the others are meant to simply display the data. A couple of nice ones are Data::Dump and Data::Printer.
my $sys = system_details;
my $log_level = $sys->{'log_level'};
my #values = #{ $log_level->{'values'} };
my $help = $log_level->{'help'};
If you need to introspect the type of structure stored in help (for example help in operating_mode is a hash, but in log_level it is a string), use the ref builtin func.
I'm using Dancer2 and the YAML session engine. I stored a complete hash in a session with the following code:
post '/login' => sub {
# ...
my $userdata = {
fname => 'John',
lname => 'Doe',
uid => 1234,
};
# ...
session userdata => $userdata;
# ...
}
The omitted code checks the login data against a database and returns that $userdata hashref.
This code creates a session file under $appdir/sessions with this content:
session file
userdata:
fname: John
lname: Doe
uid: 1234
How can I retrieve single values from this session in my app.pm file?
It works great in the template files (*.tt) and <% session.userdata.fname %> yields John, as expected.
However, I want to fetch the first name in app.pm, like so:
get '/userdetails' => sub {
my $firstname = session('userdata.fname'); # gives undef
# do sth. with $firstname
}
Is that feasable? Or do I have to
my $userdata = session('userdata'); # fetch complete hash
# do sth. with $userdata->{fname}
I tried
session('userdata.fname')
session('userdata/fname')
session('userdata:fname')
session('userdata fname')
RTFM (YAML's and Dancer2's)
but none of them worked and gave undef. The manuals and tutorials only fetch "first level values", not nested ones.
The Template Toolkit syntax is completely independent of Perl Dancer2, and you should expect any form of addressing to carry over. The origin of the data as a YAML file is also irrelevant, as the Dancer session is just a Perl hash structure
The manual doesn't make it very clear, but
session('userdata')
is the same as
session->{userdata}
So you can use
session->{userdata}{fname}
to read the subsidiary fields
(Or possibly session('username')->{fname} if you prefer, but that looks a bit icky to me!)
Note that you shouldn't use the YAML session engine in production code, as it's very slow
I have a Perl script that pulls a table from a SQL database ($row variable) and attempts to do a MongoDB update like so:
my $res = $users->update({"meeting_id" => $row[0]},
{'$set' => {
"meeting_id" => $row[0],
"case_id" => $row[1],
"case_desc" => $row[2],
"date" => $row[3],
"start_time" => $row[4],
"end_time" => $row[5],
#"mediator_LawyerID" => $row[6],
"mediator_LawyerIDs" => \#medLawIds,
"case_number" => $row[6],
"case_name" => $row[7],
"location" => $row[8],
"number_of_parties" => $row[9],
"case_manager" => $row[10],
"last_updated" => $row[11],
"meeting_result" => $row[12],
"parties" => \#partyList
}},
{'upsert' => 1}) or die "I ain't update!!!";
My client now wants ICS style calendar invites sent to their mediators. Thus, I need to know whether an update or insert happened. The documentation for MongoDB::UpdateResult implies that this is how you access such a property:
my $id = $res->upserted_id;
So I tried:
bless ($res,"MongoDB::UpdateResult");
my $id = $res->upserted_id;
After this code $id is like:
HASH(0x356f8fc)
Are these the actual IDs? If so, how do I convert to a hexadecimal string that can be cast to Mongo's ObjectId type? It should be noted I know absolutely nothing about perl; if more of the code is relevant, at request I will post any section ASAP. Its 300 lines so I didn't want to include the whole file off the bat.
EDIT: I should mention before anyone suggests this that using update_one instead of update returns the exact same result.
HASH(0x356f8fc) is a Perl Hash reference. It's basically some kind of (internal) memory address of some data.
The easiest way to get the contents is Data::Dumper:
use Data::Dumper
[...]
my $result = $res->upserted_id;
print Dumper($result);
HASH(0x356f8fc) is just the human readable representation of the real pointer. You must dump it in the same process and can't pass it from one to another.
You'll probably end up with something like
`my $id = $result->{_id};`
See the PerlRef manpage for details.
See also the MongoDB documentation about write concern.
PS: Also remember that you could use your own IDs for MongoDB. You don't need to work with the generated ones.
I have Mojolicious app with a test suite using Test::Class::Moose. Using DBIx::Class to interact with my database, is there a way to setup an in-memory database that I can add fixture data to?
I'd like to use an in memory database because it'll mean that the application will have less setup configuration. I do know how to setup an actual SQLite database for testing but managing that for testing along with a mySQL database for production doesn't sound like easy management (eg "oh no, forgot to rebuild the testing database").
Loading data from fixtures seems ideal, that way you have more control over what is actually in the database. Example, you find a bug with a row that contains certain data, add a row like that to your fixture file and test until successful.
Now how to actually set up an in-memory database using DBIx? :)
Right now this is how I'm building the schema for my Test::Class::Moose suite:
has cats => (
is => 'ro',
isa => 'Test::Mojo::Cats',
default => sub { return Test::Mojo::Cats->new(); }
);
has schema => (
is => 'ro',
lazy => 1,
builder => '_build_schema_and_populate',
);
sub _build_schema_and_populate {
my $test = shift;
my $config = $test->cats->app->config();
my $schema = Cat::Database::Schema->connect(
$config->{db_dsn},
$config->{db_user},
$config->{db_pass},
{
HandleError => DBIx::Error->HandleError,
unsafe => 1
}
);
require DBIx::Class::DeploymentHandler;
my $dh = DBIx::Class::DeploymentHandler->new({
schema => $schema,
sql_translator_args => { add_drop_table => 0 },
schema_version => 3,
});
$dh->prepare_install;
$dh->install;
my $json = read_file $config->{fixture_file};
my $fixtures = JSON::decode_json($json);
$schema->resultset($_)->populate($fixtures->{$_}) for keys %{$fixtures};
return $schema;
}
Where my config specifies dbi:SQLite:dbname=:memory: as the database dsn.
When running the test suite, the tables don't seem to be loaded, as I get errors stating the table does not exist, eg Can't locate object method "id" via package "no such table: cats"
Is there some extra setup that I'm not doing when wanting to deploy to an in-memory database?
Thanks
PS:
Doing the following works in a single script, I don't know if I'm doing something that Test::Class::Moose or Mojo doesn't like with the above
#!/usr/bin/perl
use Cats::Database::Schema;
use File::Slurp;
use JSON;
use Data::Dumper;
my $schema = Cats::Database::Schema->connect(
'dbi:SQLite:dbname=:memory:', '', ''
);
my $json = read_file('../t/fixtures.json');
my $fixtures = JSON::decode_json($json);
$schema->deploy();
$schema->resultset($_)->populate($fixtures->{$_}) for keys %{$fixtures};
# returns fixture data fine
# warn Dumper($schema->resultset('User')->search({}));
I believe I figured it out
The way I use the DBIx schema in the app is to instantiate it within a base controller which all sub controllers inherit. No matter how I built and populated the in memory database in the Test::Class::Moose object, it would not be using the instance specified there, instead it would be using the one specified in the base controller.
the solution was to move the schema construction up one level (from controller to the app root) as an attribute, allowing me to override it in Test Mojo to use the in memory db.
Hi I am prepopulating the db as said here
http://docs.rhomobile.com/faq#how-to-pre-populate-client-database
but I have a problem, that when I make reset DB with default code
def do_reset
Rhom::Rhom.database_full_reset
SyncEngine.dosync
#msg = "Database has been reset."
redirect :action => :index, :query => {:msg => #msg}
end
then I am losing the data. How can I make that the prepopulated database alsways will be loaded, when I make reset.
Cheers
I come up with such solution
in view do_reset.erb
<%
Antwort.delete_all()
file_name = File.join(Rho::RhoApplication::get_model_path('app','Settings'), 'antwort.txt')
file = File.new(file_name,"r")
aid=0
file.each_line("\n") do |row|
col = row.split("|")
aid=aid+1
#antwort=Antwort.create(
{"aid" => aid, "qid" => col[0],"antwort"=>col[1],"richtig"=>col[2]}
)
qty=file.lineno
break if file.lineno > 3000
end
Questions.delete_all()
file_name = File.join(Rho::RhoApplication::get_model_path('app','Settings'), 'questions.txt')
file = File.new(file_name)
file.each_line("\n") do |row|
col = row.split("|")
#question=Questions.create(
{"id" => col[0], "question" => col[1],"answered"=>'0',"show"=>'1',"tutorial"=>col[4]}
)
break if file.lineno > 1500
end
file.close
#msg="OK"
%>
But only problem I have now is single quotes aka ' in the texts. They are then displayed in app as a triangle with ? inside like � one. What to do?
You can seed the database using a pipe delimited text file, as explained here.
So, in the instance you are using the Property Bag model definitions, you will have a file called object_values.txt and load up whichever sources, properties, and values necessary.