I have cities table and trying to insert city, upon database creation. The table structure is pretty simple, it has just id and name column.
In onCreate method of my database class, I create table with this command:
var tblCities = 'cities';
await db.execute('CREATE TABLE $tblCities (id INTEGER PRIMARY KEY, name TEXT)');
I have Cities model class with fromMap and toMap methods.
There are about 350 cities, and I wanted to insert them in the table.
Q. What is the best and easy way to do that?
This comes in my mind:
creating list of city
using for loop to iterate entire list
creating map of the city using toMap method
calling db.insert method inside the loop
I'm not sure, but this seem dumb approach so thinking about better and optimized solution...
As mentioned by #chunhunghan, you can use batch to insert bulk data.
Here's step by step guideline:
Get ready your json file e.g cities.json (create csv file of data and use csv to json converter like this)
Add cities.json file in your assets directory
Define it in pubspec.yaml like this:
assets:
- assets/cities.json
Paste this code inside onCreate method of your database class (make sure its after table creation query)
Batch batch = db.batch();
String citiesJson = await rootBundle.loadString('assets/json/cities.json');
List citiesList = json.decode(citiesJson);
citiesList.forEach((val) {
//assuming you have 'Cities' class defined
Cities city = Cities.fromMap(val);
batch.insert(tblCities, city.toMap());
});
batch.commit();
That's it! :)
There is Batch support
To avoid ping-pong between dart and native code, you can use Batch:
batch = db.batch();
batch.insert('Test', {'name': 'item'});
batch.update('Test', {'name': 'new_item'}, where: 'name = ?', whereArgs: ['item']);
batch.delete('Test', where: 'name = ?', whereArgs: ['item']);
results = await batch.commit();
official example https://github.com/tekartik/sqflite/blob/master/sqflite/example/lib/batch_test_page.dart
In your case, for loop list with batch.insert command, it's easier to maintain
for simplicity syntax, use toMap, example
batch.insert("cities", city.toMap());
detail https://www.techiediaries.com/flutter-sqlite-crud-tutorial/
If you prefer rawInsert, please reference Insert multiple records in Sqflite
You can write a raw query to insert all the data at once into the database.
Related
This is the simplified structure of my tables - 2 main tables, one relation table.
What's the best way to handle an insert API for this?
If I just have a Client and Supabase:
- First API call to insert book and get ID
- Second API call to insert genre and get ID
- Third API call to insert book-genre relation
This is what I can think of, but 3 API calls seems wrong.
Is there a way where I can do insert into these 3 tables with a single API call from my client, like a single postgres function that I can call?
Please share a general example with the API, thanks!
Is there any reason you need to do this with a single call? I'm assuming from your structure that you're not going to create a new genre for every book you create, so most of the time, you're just inserting a book record and a book_gen_rel record. In the real world, you're probably going to have books that fall into multiple genres, so eventually you're going to be changing your function to handle the insert of a single book along with multiple genres in a single call.
That being said, there are two ways too approach this. You can make multiple API calls from the client (and there's really no problem doing this -- it's quite common). Second, you could do it all in a single call if you create a PostgreSQL function and call it with .rpc().
Example using just client calls to insert a record in each table:
const { data: genre_data, error: genre_error } = await supabase
.from('genre')
.insert([
{ name: 'Technology' }
]);
const genre_id = genre_data[0].id;
const { data: book_data, error: book_error } = await supabase
.from('book')
.insert([
{ name: 'The Joys of PostgreSQL' }
]);
const book_id = book_data[0].id;
const { data: book_genre_rel_data, error: book_genre_rel_error } = await supabase
.from('book_genre_rel_data')
.insert([
{ book_id, genre_id }
]);
Here's a single SQL statement to insert into the 3 tables at once:
WITH genre AS (
insert into genre (name) values ('horror') returning id
),
book AS (
insert into book (name) values ('my scary book') returning id
)
insert into book_genre_rel (genre_id, book_id)
select genre.id, book.id from genre, book
Now here's a PostgreSQL function to do everything in a single function call:
CREATE OR REPLACE FUNCTION public.insert_book_and_genre(book_name text, genre_name text)
RETURNS void language SQL AS
$$
WITH genre AS (
insert into genre (name) values (genre_name) returning id
),
book AS (
insert into book (name) values (book_name) returning id
)
insert into book_genre_rel (genre_id, book_id)
select genre.id, book.id from genre, book
$$
Here's an example to test it:
select insert_book_and_genre('how to win friends by writing good sql', 'self-help')
Now, if you've created that function (inside the Supabase Query Editor), then you can call it from the client like this:
const { data, error } = await supabase
.rpc('insert_book_and_genre', {book_name: 'how I became a millionaire at age 3', genre_name: 'lifestyle'})
Again, I don't recommend this approach, at least not for the genre part. You should insert your genres first (they probably won't change) and simplify this to just insert a book and a book_genre_rel record.
I have just started to use Moor Database for Flutter. I am going to join my two tables to get some columns from both tables.
I have checked the example that is given in docs as follow:
// we define a data class to contain both a todo entry and the associated category
class EntryWithCategory {
EntryWithCategory(this.entry, this.category);
final TodoEntry entry;
final Category category;
}
// in the database class, we can then load the category for each entry
Stream<List<EntryWithCategory>> entriesWithCategory() {
final query = select(todos).join([
leftOuterJoin(categories, categories.id.equalsExp(todos.category)),
]);
// see next section on how to parse the result
}
I am not able to understand that where to put this class. If I am creating a new class then it's giving me an error that the select keyword is not found. Also tried to import related to moor but not working.
Where I can write join queries and make this class?
Moor basically says to get the results and build the class manually. That class has no relationship with the database, so you can put it wherever you want. It is just the suggested way of doing it.
So, the select statement returns an object that you can iterate over the resulting rows, as an SQL response. And with that results build the classes that will be returned.
Look at the next example of the docs:
return query.watch().map((rows) {
return rows.map((row) {
return EntryWithCategory(
row.readTable(todos),
row.readTableOrNull(categories),
);
}).toList();
});
Because is a stream, calls watch(),and then map().
This first map returns the result of the query, properly name rows, every time one of the rows changes in the database, it will emit all the rows again.
The second map inside the first is for turning every row into a EntryWithCategory object. That way the whole function returns a list of those object updated with every change.
You can create other model for several tables.
Try this variant.
import 'package:drift/drift.dart';
part 'car_dao.g.dart';
#DriftAccessor(tables: [Cars, Bikes])
class CarDao extends DatabaseAccessor<AppDatabase> with _$CarDaoMixin {
final AppDatabase db;
CarDao(this.db) : super(db);
Future<List<CarWithBikeModel>> getCarsWithBikes() async {
final carList = await (select(cars).get();
final bikeList = await (select(bikes).get();
return CarWithBikeModel(
cars: carList,
bikes: bikeList);
}
}
List list = new List();
I have a list of Guid. What is the best to check all guid exits or not using ef core table?
I am currently using the below code but the performance is very bad. assume user table as 1 million records.
for Example
public async Task<bool> IsIdListValid(IEnumerable<int> idList)
{
var validIds = await _context.User.Select(x => x.Id).ToListAync();
return idList.All(x => validIds.Contains(x));
}
The performance is bad because you are reading each row of the table into memory, and then iterating through it (ToList materializes the query.) Try using the Any() method to take advantage of the strength of the database. Use something like the following: bool exists = _context.User.Any(u => idList.Contains(u));. This should translate to an SQL IN clause.
Provided you assert that the # of IDs being sent in is kept reasonable, you could do the following:
var idCount = _context.User.Where(x => idList.Contains(x.Id)).Count();
return idCount == idList.Count;
This assumes that you are comparing on a unique constraint like the PK. We get a count of how many rows have a matching ID from the list, then compare that to the count of IDs sent.
If you're passing a large # of IDs, you would need to break the list up into reasonable sets as there are limits to what you can do with an IN clause and potential performance costs as well.
I am creating a collection of judges and courthouses. Every judge will be assigned to one courthouse. I have set up my relation to be that courthouse has many judges
I am attempting to do this programmatically when the app loads. I have a function that is able to populate all the fields in judge except the relation to courthouse. My function uses the Strapi API like this
const judge = await strapi.query('judge').create({
name: data[i].name,
},
{
courthouse: data[i].courthouse_name // here is where I think the relation is created
}
)
I am passing in a string that has the name of courthouse, because I don't know the ID of the courthouse in the Courthouse collection.
My question is it possible to create a relation to another collection by anything other than an ID? How can I create a relation to a courthouse by its name?
I couldn't find a way around building a relationship between two models without the ID, so I created a custom solution using the Strapi lifecycle hooks
Essentially what I did I utilized the beforeCreate lifecycle hook to query and find the courthouse that matches the name like this:
// judges.js
async beforeCreate(result, data) {
const courthouse = await strapi.query('courthouse').find(
{courthouse_name:data.courthouse}
); // returns the courthouse that matches the name
result['courthouse'] = courthouse[0].id; // populates the relational field with the
// ID of the courthouse
}
The response object contained the courthouse's ID and I manipulated the data that is being sent to the create command like this:
const judge = await strapi.query('judge').create({
name: data[i].name,
courthouse: data[i].courthouse_name
})
The result is an object that looks like this:
{name: 'Garfield Lucas, courthouse: 7463987}
I have a Postgres DB and using ObjectionJS as my ORM.
I have the following ObjectionJS code to patch two columns of a user_system table.
public static async DeleteSystem(systemId: number, context: Context): Promise<UserSystem> {
const system = await UserSystem.query().context(context).patch({
state: PointInTimeState.inactive,
receiveNotifications: false
}).where({
system_id: systemId
}).throwIfNotFound().debug()
//("as any" for now because the system variable is a number at this point (i.e NOT of type Promise<UserSystem> as in the signature)
return system as any
}
Questions
Is there a way in which I could return all the Rows that were not affected by this patch?
If so, how, without having to write two separate queries (to update and then requery new data) to the back end ?
As far as I know writing CTE combining two queries to single is the only way.
With objection / knex they can be done with https://knexjs.org/#Builder-with
So in with query you do the .patch(...).where(...).returning('id') and in main query you select all rows from the table, which are not in id set returned by the first query.
Something like this (not sure if this works at all like this with objection):
UserSystem.query().context(context).with('patchedIds',
UserSystem.query().patch({
state: PointInTimeState.inactive,
receiveNotifications: false
}).where({
system_id: systemId
}).returning('id')
).whereNotIn('id', builder => {
builder.from('patchedId').select('id')
})