Recommended data flow (import/export) - flutter

I have a flutter application which (simply put) list some data on various screens and can be modified. My current data approach works, but I feel it is not a best practice or optimal.
Currently, when a object is saved, it is converted to JSON (using dart:convert) and stored in a file on the device (using dart.io), overriding the file if it exist. Every screen that needs to display these objects reads the file to get the objects. Every time there is a change that needs to be saved, it exports everything (overwrites) again then imports it again to display it.
The reason I chose JSON over S is because I want to add a web portion later. Does this approach of reading/writing a best practice? I feel this much reading/writing of all the data for most screens could cause some performance issues.
Any advice is appreciated!

This is a possible way to keep data in-memory and write to disk when changes are made to your datamodel/settings.
I use RxDart myself. You don't need it per se, although it does make life easier. I'll be simplifying the examples below, so you get to know the concept and apply it to your own needs.
Let say you keep track of data in your settings class:
#JsonSerializable()
class Settings {
String someData1;
String someData2;
// json seriazable functions
}
You need a "handler"1 or something similar that manages changes made to your Settings and also to read/write data:
class SettingsHandler {
Settings _settings;
StreamController<Settings> _settingsController = BehaviorSubject<Settings>();
StreamController<String> _data1Controller = BehaviorSubject<String>();
StreamSink<String> get data1Input => _data1Controller.sink;
Observable<String> get data1Output => Observable(_data1Controller.stream);
Future<Settings> _readFromDisk() async {
// do your thing
}
Future<Settings> _writeToDisk(Settings settings) async {
// do your thing
}
Future<void> init() async {
// read from disk
_settings = await _readFromDisk();
_settingsController.sink.add(_settings);
// initialize data
data1Input.add(_settings.someData1);
data1Output
.skip(1) // we skip because we just added our initialization data above.
.listen((value) =>
// we must propagate through the update function
// otherwise nothing gets written to disk
update((settings) => settings.someData1 = value)
);
// when changes are made, it needs to notify this stream
// so everything can be written to disk
_settingsSaver.output
// save settings every 2.5 seconds when changes occur.
.debounceTime(const Duration(milliseconds: 2500))
// get the changes and write to disk
.listen((settings) => _writeToDisk(settings));
}
// this function is crucial as it allows changes to be made via the [adjustFunc]
// and then propagates this into the _settingsSaver stream.
void update(void Function(Settings input) adjustFunc) {
adjustFunc(_settings);
_settingsSaver.sink.add(_settings);
}
}
So now you can do something like
var handler = SettingsHandler();
await handler.init();
// this
handler.data1Input.add('NewData');
// or this
handler.update((settings) {
settings.someData1 = 'NewData';
});
Remember, this code only shows how the concept can work. You need to change it for your situation. You could also decide to not expose data1Input or the update(...) function, this is up to your own design.
1 I personally use BloC, your situation might require a different way.

Related

Global variables or singletons lose their value (reset) after refresh

I have a simple webpage that I would like to refresh every few minutes, defined by the user. So, for that I thought of using a global variable to store the value of how often the page should be refreshed. Unfortunately, it seems that after the page refreshes, all of the variable values are lost/restarted for some reason.
I tried using different kinds of imports, the singleton pattern, service pattern, a simple static variable inside a "Globals" class but nothing seems to work.
My current, relevant code for the global class is this:
class Globals {
static final Globals _instance = Globals._internal();
factory Globals() => _instance;
Globals._internal() {
_GLOBAL_REFRESH_MINUTES = 0;
}
int? _GLOBAL_REFRESH_MINUTES;
int get REFRESH_MINUTES => _GLOBAL_REFRESH_MINUTES!;
set REFRESH_MINUTES(int value) => _GLOBAL_REFRESH_MINUTES = value;
}
import 'package:myproject/common/globals.dart';
Globals _globals = Globals();
if (_globals.REFRESH_MINUTES > 0) {
new Timer.periodic(Duration(minutes: _globals.REFRESH_MINUTES),
(Timer t) => html.window.location.reload());
}
To solve your problem you need to save data to the browser, I could suggest hive: https://pub.dev/packages/hive this is great for things like 'remember me' on login or user preferences, configurations that need to persist across browser sessions.
Without knowing why you are refreshing the browser every few minutes, is it possible that another solution which does not globally restart the application could be leveraged? Perhaps whatever needs restarting could be disposed and reinitialized in its own class for example. If so that would be the correct approach to take.

Add or remove item in datagrid does not trigger WhenAnyPropertyChanged

I am using dynamic data with reactiveui,
` _propList.Connect()
.WhenAnyPropertyChanged()
.Subscribe(t =>
{
}`
the code will be trigger if I just edit any item in the grid. However, when I try to add or remove an item, it is not triggered.
In my view model I have something like this
private SourceList<Decision> _myList { get; set; } = new SourceList<Decision>();
private readonly IObservableCollection<Decision> _targetCollection = new ObservableCollectionExtended<Decision>();
public IObservableCollection<Decision> TargetCollection => _targetCollection;
in my view, I simply
this.OneWayBind(VM, vm => vm.TargetCollection, v => v.DataGrid1.DataSource);
If I remove or Add item in the grid, and press Save
_myList.Count() didn't change, but
_TargetCollection.Count() will increase or decrease by number of items I delete
In my ViewModel
OKCmd = ReactiveCommand.Create(() =>
{
//// _myList.Connect()
////.Subscribe(t =>
//// {
//// ;
//// }
//// );
t.Items.count() and it is the initial load items, but I couldn't seem to know what items have been added or removed. Am I missing something.
Of course, I can keep track of what items are added or removed in the UI, but I am hoping I don't have to do that.
Thanks.
To help me answer your question, I need to better understand what you are trying to achieve but first I will explain what the default behaviour of DD is.
If you want add / remove events you need _propList.Connect().Subscribe(changes => ...). These are the collection changes and you will receive all collection change events including the initial load, but no inline changes.
By default, no property changes are wire in. This is because to monitor property changes is expensive and is opt in only. Also WhenAnyPropertyChanged() never tiggers for the initial load. This is because the item is already loaded and no properties have changed between Connect being called and the property changed observable being subscribed to.
Following on from 2, you will never receive a property changed when an item is removed from the underlying source. This is because when an item it removed, any inline subscriptions are disposed of. Otherwise there would be memory leaks.
Another option for monitoring inline changes is to make use of 'MergeMany' which allows you to craft any observable on a specific item, and in your case you can create an observable to return the initial value as well as as subsequent changes.
It is possible using standard rx to listen to collection changes and inline changes in a single observable, which you would have to compose yourself. For example
var myCollectionChanges = _propList.Connect();
var myPropertyChanges = _propList.Connect().WhenAnyPropertyChanged();
var allMyChanges = myCollectionChanges.Select(_ => Unit.Default)
.Merge(myPropertyChanges.Select(_ => Unit.Default));
In the this example, I have used Select(_ => Unit.Default) to enable the merge operator as it requires the same signature. However what signature is returned is up to you, the key point being that the signatures must match.

Separating classes

I have a class called UFDevice, in order to initialise it needs a location string.
I also have a class called UFResponse which among other things provides a location.
As the device only requires a location should I just take that in, so that it could be init'ed in some use case where there is no UFResponse.
Or should I pass in the whole response, in case later on it needs more info than just the location?
in pseudocode:
foundDevice(Data data) {
response = new UFResponse(data);
device = new UFDevice(response);
}
or:
foundDevice(Data data) {
response = new UFResponse(data);
device = new UFDevice(response.location);
}
or even should I encapsulate UFResponse in UFDevice, as currently it's only used to create UFDevices:
foundDevice(Data data) {
device = new UFDevice(data);
}
Future possibilities could include:
//maybe in the future I have saved a favourite location so need to do:
loadFavourite(String location) {
device = new UFDevice(location);
}
//or device needs more info
device = new UFDevice(location, color, ...20 more parameters...);
Where do I draw the line of separation? More importantly how can I decide this for myself in the future?
It sounds like a problem of interface segragation (https://en.wikipedia.org/wiki/Interface_segregation_principle). UFDevice is constructed from a UFResponse, but it doesn't need everything the UFResponse contains. It only needs a part of it, and you don't want UFDevice to be affected when UFResponse is changing in areas that should not affect UFDevice.
One approach is to have UFResponse inherit from an interface called UFDeviceParams, if this makes sense (might be multiple inheritance), and then UFDevice should get in its constructor a reference to UFDeviceParams.
This allows to initialize UFDevice based on the entire UFResponse, or based on a more light-weight instance of UFFavouriteParams (that also inherits from UFDeviceParams) that contains only the location + color etc...
foundDevice(Data data) {
response = new UFResponse(data);
device = new UFDevice(response);
}
loadFavourite(String location) {
params = new UFFavouriteParams(location);
device = new UFDevice(params);
}
To really know if this is the best approach for your case, one would need to learn more about your system, understand the use cases and the boundaries between modules. I recommend to watch Robert Martin's video on the Interface Segratation Principle and SOLID principles in general (https://cleancoders.com/category/solid-principles)

Good way to repeat a test, inserting an extra action?

I like the way Catch has nested hierarchies of tests, and it works through the combinations. It feels more natural than the setup/teardown of xUnit frameworks.
I now have a set of tests. What I want to do, about halfway down is insert a load/save serialization test, and then repeat all the tests below that point, first without the load/save, then again using the data it loaded from the serialization process. I.e. to prove that the load/save was correct.
I cannot get my head around if Catch has anything that can help with this? If it was phpUnit, I would be thinking about a string of #depends tests, and use a #dataProvider with a boolean input. A bit ugly.
(If that does not make sense, let me know, and I'll try to work out a minimal example)
The issue here is that Catch is designed to descend a tree-like organisation of tests and it automatically discovers all of the leaf-nodes of the structure and calls back into the test cases with untested code paths until they're all tested. The leaf nodes (tests, sections) are meant to be independent.
It sounds like you want to test a repository - something that can persist some data and then load it back in.
To repeat the exact same tests in two different scenarios (before serialisation, after serialisation) you'd need to put the same tests into some common place and call into that place. You can still use the same Catch macros in a non-test-case function, as long as you call it from a test case.
One possible way to do this is:
struct TestFixture {
Data data;
Repository repository;
TestFixture() : data(), instance() { }
};
void fillUpData(Data& data) {
// ...
}
void isDataAsExpected(Data& data) {
// Verify that 'data' is what we expect it to be, whether we
// loaded it or filled it up manually
SECTION("Data has ...) {
REQUIRE(data...);
}
}
TEST_CASE_METHOD(TestFixture, "Test with raw data") {
fillUpData(data);
isDataAsExpected(data);
REQUIRE(repository.save(data));
}
TEST_CASE_METHOD(TestFixture, "Operate on serialised data") {
REQUIRE(repository.load(data));
isDataAsExpected(_data);
}
One possible alternative is to supply your own main and then use command-line arguments to control whether/not the data is first serialised.
There's a third way I can think of that uses a non-quite-ready-yet feature of Catch - Generators:
TEST_CASE("...") {
using Catch::Generators;
int iteration(GENERATE(values(0, 1)));
const bool doSave(iteration == 0);
const bool doLoad(iteration == 1);
Repository repository;
Data data;
if (doLoad) {
REQUIRE(repository.load(data));
} else {
// fill up data
}
REQUIRE(..data..test..);
if (doSave) {
REQUIRE(repository.save(data));
}
}
The advantage of this method is you can see the flow and the test runs twice (for the two values) but the major disadvantage is that Generators are incompatible with SECTIONs and BDD-style features.

.NET Rx - ReplaySubject buffer size not working

I've been using .NET Reactive Extensions to observe log events as they come in. I'm currently using a class that derives from IObservable and uses a ReplaySubject to store the logs, that way I can filter and replay the logs (for example: Show me all the Error logs, or show me all the Verbose logs) without losing the logs I've buffered.
The problem is, even though I've set a buffer size on the subject:
this.subject = new ReplaySubject<LogEvent>(10);
The memory usage of my program goes through the roof when I use OnNext to add to the observable collection on an infinite loop:
internal void WatchForNewEvents()
{
Task.Factory.StartNew(() =>
{
while (true)
{
dynamic parameters = new ExpandoObject();
// TODO: Add parameters for getting specific log events
if (this.logEventRepository.GetManyHasNewResults(parameters))
{
foreach (var recentEvent in this.logEventRepository.GetMany(parameters))
{
this.subject.OnNext(recentEvent);
}
}
// Commented this out for now to really see the memory go up
// Thread.Sleep(1000);
}
});
}
Does the buffer size on ReplaySubject not work? It doesn't seem to be clearing the buffer when the buffer size is reached. Any help much appreciated!
UPDATE:
I add subscribers like this (Is this wrong?):
public IDisposable Subscribe(IObserver<LogEvent> observer)
{
return this.subject.Subscribe(observer);
}
...which is called like:
// Inserts into UI ListView
this.logEventObservable.Subscribe(evt => this.InsertNewLogEvent(evt));
I'm not sure if this is the definitive answer, but I suspect that you're hitting an issue because of concurrency around the scheduler you're using. The constructor you're calling on ReplaySubject looks like this:
public ReplaySubject(int bufferSize)
: this(bufferSize, TimeSpan.MaxValue, Scheduler.CurrentThread)
{ }
The Scheduler.CurrentThread worries me. Try changing it to Scheduler.ThreadPool and see if that helps.
Also, as a side note, you seem to be mixing Rx with TPL and old fashioned thread sleeping. It's usually best to avoid doing that. You could change your WatchForNewEvents code to look like this:
dynamic parameters = new ExpandoObject();
var newEvents =
from n in Observable.Interval(TimeSpan.FromSeconds(1.0))
where this.logEventRepository.GetManyHasNewResults(parameters)
from recentEvent in
this.logEventRepository.GetMany(parameters).ToObservable()
select recentEvent;
newEvents.Subscribe(this.subject);
That's a nice compact Rx-y way of doing things.