I have my webservice running on dropwizard. I am using the slf4j MDC to set
various attributes in my webservice. Currently I have 2 service endpoints service1 and service2. Each of them calls a function func() where the attributes are logged using MDC. But when I later retrieve the MDC to log these attributes in a buffer appender, only service1's attributes are available in the MDC, service2's MDC context returns null. I am not clearing the MDC anywhere in my code. What could be the reason.
service1()
{func()
}
service2()
{
func()
}
func() {
MDC.put(x,y);
}
I found out what the issue was. Although the func() being called is the same. service2() invokes the function in a separate thread pool. (Therefore the MDC dies within that thread pool. Whereas service1 invokes func() in the same thread, therefore it lives on and the attributes set in the context are available.
MDC's scope is within the thread.
Related
Long, long ago, when we were using mgo.v2, we created some wrapper functions that copied the session, set the read pref and returned that for consumption by other libraries, e.g.
func NewMonotonicConnection() (conn *Connection, success bool) {
conn := &Connection{
session: baseSession.Copy(),
}
conn.session.SetMode(mongo.Monotonic, true)
return conn, true
}
We now just pass the default client (initialized using mongo.Connect and passed into a connection singleton) in an init function and then consumed like this:
func NewMonotonicConnection() (conn *Connection, success bool) {
conn = defaultConnection
return conn, true
}
My understanding is that to leverage connection pooling, you need to use the same client (which is contained in defaultConn), and session is now implicitly handled inside of the .All()/cursor teardown. Please correct me if I'm wrong here.
It would be nice if we could still set the readpref on these connections (e.g. set NearestMode on this connection before returning), but what's the community/standard way of doing that?
I know I could call mongo.Connect over and over again, but is that expensive?
I could create different clients - each client with a different readpref - but I was thinking that if a write occurred on that connection, it wouldn't ever go back to reading from a slave.
It looks like I *can create sessions explicitly, but I'm not certain I should or if there are any implications around managing those explicitly in the new driver.
There are a couple things I learned on this quest through the mongo-go-driver codebase that I thought I should share with the world before closing this question. If I'm wrong here - please correct me.
You should not call Connect() over and over if you want to leverage connection pooling. It looked like each time Connect() was called, a new socket was created. This means that there's a risk of socket exhaustion over time unless you are manually defer Close()-ing it each time.
In mongo-go-driver, sessions are automatically handled under the covers now when you make the call to execute the query (e.g. All()). You can* explicitly create and teardown a session, but you can't consume it using the singleton approach I proposed above without having to change all the caller functions.
This is because you can no longer call query operations on the session, you instead have to consume it using a WithSession function at the DB operation itself
I realized that writeconcern, readpref and readconcern can all be set at the:
client level (these are the defaults that everything will use if not overridden)
session level
database level
query level
So what I did is create Database options and overloaded *mongo.Database e.g.:
// Database is a meta-helper that allows us to wrap and overload
// the standard *mongo.Database type
type Database struct {
*mongo.Database
}
// NewEventualConnection returns a new instantiated Connection
// to the DB using the 'Nearest' read preference.
// Per https://github.com/go-mgo/mgo/blob/v2/session.go#L61
// Eventual is the same as Nearest, but may change servers between reads.
// Nearest: The driver reads from a member whose network latency falls within
// the acceptable latency window. Reads in the nearest mode do not consider
// whether a member is a primary or secondary when routing read operations;
// primaries and secondaries are treated equivalently.
func NewEventualConnection() (conn *Connection, success bool) {
conn = &Connection{
client: baseConnection.client,
dbOptions: options.Database().
SetReadConcern(readconcern.Local()).
SetReadPreference(readpref.Nearest()).
SetWriteConcern(writeconcern.New(
writeconcern.W(1))),
}
return conn, true
}
// GetDB returns an overloaded Database object
func (conn Connection) GetDB(dbname string) *Database {
dbByName := &Database{conn.client.Database(dbname, conn.dbOptions)}
}
This allows me to leverage connection pooling and maintain backwards compatibility with our codebase. Hopefully this helps someone else.
Right now i'm having a situation where i'm starting several request from separated threads, something like a Thread Pool.
I want to make that requests to be synchronous, so they will be started and ended on exactly the same thread, not the Main thread.
I tried to set the .observeOn(CurrentThreadScheduler.instance), but as the response handler thread was always Main thread instead of a current.
Is it possible to specify the current thread as a handler thread for the response?
Is it possible to specify the current thread as a handler thread for
the response?
There is a way! Take a look at MoyaProvider class.
/// Request provider class. Requests should be made through this class only.
open class MoyaProvider<Target: TargetType>: MoyaProviderType {
/// Propagated to Alamofire as callback queue. If nil - the Alamofire default (as of their API in 2017 - the main queue) will be used.
let callbackQueue: DispatchQueue?
/// Initializes a provider.
public init(..., callbackQueue: DispatchQueue? = nil, ...) {
...
self.callbackQueue = callbackQueue
}
}
You are allowed to provide a callback queue. Just provide the desired queue.
I have a struct that abstracts MongoDB for specific objects in my system, and during the factory function for it I set a few variables like hostname and other connection information. At the moment in each of the UserService's methods* it creates a new session, does what it needs to, then closes the session when it's done.
The system itself is the backend of a web application, so the lifetime of each UserService struct is usually only a few seconds at most, but more than one operation might be performed for each request that comes in. So I'm wondering whether to make session a member of the struct, initialise it when I create my UserService, and then ensure the request handler function that creates it calls a close function once it's done all the operations it needs to. I haven't been able to find anything that encourages or discourages this behaviour so I'm not really sure if the mgo package I'm using is designed that way.
This is my proposed code:
type UserService struct {
callingUserId id.ID //The ID of the user calling this service.
host string //The host name.
session mgo.Session //The active session.
}
func NewUserService(userId id.ID) *UserService {
nus := new(UserService)
nus.callingUserId = userId
nus.host = "localhost" //TODO change this to read from file or buffer.
nus.session, sessionErr = mgo.Dial(nos.host)
if sessionErr != nil {
//TODO log error.
}
return nus
}
func (us UserService) GetById(usrId id.ID) *users.User {
//TODO get a user from the database with the matching ID using the existing session.
}
There will also be a function called UserService.CloseSession, which as I mentioned, will be called after all the required operations are done by the handler function that creates the service.
Does anyone know if this is a sensible idea or should I just call mgo.Dial in every function on UserService?
Opening a connection to MongoDB or any other DB is an expensive task that should be done once and kept operative as much as possible inside your service.
You have three options:
A local package level MongoDB instance.
Using Contexts and Composing Handlers.
A method receiver which holds the MongoDB Instance.
This is a very informative blog post for handling DBs connections.
Take a look to this example, especially for MongoDB
In my app client uses services backed by Observables. Each service call is session based, so that a session has to be started before a business-service method can be called.
In order to init session I made an Observable that does it. My problem is that whenever client uses more than one business-service in parallel, session initialization gets duplicated. Client code is session-agnostic. I need a way to init session in such a way that the session observable only gets called once(when the first business-service method gets called). Basically all subsequent business-observers have to wait for condition(session initialization). Can you draw a pattern for it?
Client code would look like:
protected void onCreate(Bundle savedInstanceState) {
itemService.getItems(10).subscribe(new Observer<List<Item>>() {..});
userService.getProfile().subscribe(new Observer<List<Profile>>() {..});
}
While there're 2 calls, I need to make the session Observable execute once only and make business Observables wait for the session initialization to complete and then start doing its' job.
Thanks.
If the session initialization is an Observable, then you can delay all other Observables that depend on it, using the delay operator with the session initialization Observable as parameter: http://reactivex.io/RxJava/javadoc/rx/Observable.html#delay%28rx.functions.Func1%29
Since you already have a session Observable implemented, one possible solution would be to convert it to an AsyncSubject. It would execute only once and would emit only the last item (or only item) to each subscriber.
Depending on how your session initialization code works, it may be more elegant to use one of the Async operators such as Async.start() or Async.startFuture() to create your session observable.
I have a multithreaded Scala application and I have a shutdown hook that calls a shutdown() method on an object. Unfortunately this doesn't seem to be reliable. Many times it fails to handle SIGINT and throws a NoClassDefFoundError on an anonymous function.
How do I make this shutdown handler more robust?
One suggestion: Rather than registering shutdown() hook directly, create a ShutdownManager object and have it register a shutdown() hook. Your application objects add themselves to the ShutdownManager which can then call some application object shutdown method, either using a trait or through a function object.
preload all the classes your hook needs with either:
Class.forName("YourObject") or Class clasz = YourObject.class