Associate an object with a buffer - emacs

Question

What I'm trying to do is very similar to what make-local-variable does except that I don't want variables to be declared for every buffer created (only for those which belong to a particular mode). This is mostly a performance concern, the objects created may be big. AND more importantly I'd need a function like (get-buffer-property buffer property-name) to figure out what is the state of the particular buffer.
Essentially, I'm after something similar to get-buffer-proccess except that I need something other than a process.
Will it make sense to create such objects in the declaration of the major mode and destructing them in a hook for when such buffer is killed, or is there a better way for that?
This is what I have so far:
(defun haxe-get-buffer-property (buffer property)
"Pops to BUFFER, reads the value of the PROPERTY and returns it."
(let ((result
(save-excursion
(pop-to-buffer buffer)
(symbol-value property))))
result))
(defmacro haxe-buffer-property (buffer property)
`(haxe-get-buffer-property ,buffer ',property))
(defmacro deflocal (var &rest body)
(let ((symb var)
(val (car body))
(doc (cadr body)))
`(progn
(defvar ,symb nil ,doc)
(unless ,symb (setq ,symb ,val))
(make-local-variable ',symb))))
But I don't like that I have to visit buffers because I'm not sure of side effects.
EDIT: Some more info.
What happens is like so, there are multiple buffers which can interact with a network connection process. This process can be shared by groups of buffers (it is important that buffers share this process), but it also may happen that there exist simultaneously multiple buffers that have different processes assigned to each other. Besides the process itself, there's a lot of info to store about the state of the process (how much data was received, what was sent, errors etc.) Similarly this data has to be shared by groups of buffers.
EDIT2:
This is what become of the code above, just in case anyone will need it.
(defmacro deflocal (var &rest body)
(let ((symb var)
(val (car body))
(doc (cadr body)))
`(progn
(set (make-local-variable ',symb) ,val)
(put ',symb 'variable-documentation ,doc))))
(defun haxe-get-buffer-property (buffer property)
"Pops to BUFFER, reads the value of the PROPERTY and returns it."
(let ((result
(with-current-buffer buffer
(symbol-value property))))
result))
(defun haxe-set-buffer-property (buffer &rest proplist)
"Pops to BUFFER and sets properties in parallel, similar to `pset'."
(let ((result
(with-current-buffer buffer
(loop for (property value) on proplist by #'cddr
do (set property value)
finally (return value)))))
result))
(defmacro haxe-buffer-pset-property (buffer &rest proplist)
`(haxe-set-buffer-property
,buffer
,#(loop for (key value) on proplist by #'cddr
nconc (list (list 'quote key) value))))
(defalias 'haxe-pbset #'haxe-buffer-pset-property)
(defmacro haxe-buffer-setf-property (buffer &rest proplist)
`(with-current-buffer ,buffer
,#(list (append '(setf) proplist))))
(defalias 'haxe-pbsetf #'haxe-buffer-setf-property)

Answer 1

On an emacs -q session I executed the following:
;; set up a hook to create buffer local variable
(add-hook 'html-mode-hook
(lambda ()
(set (make-local-variable 'my-var) "abcd")))
;; find two files, one html and another java
(find-file-noselect "X.java")
(find-file-noselect "X.html")
;; see the buffer value in html buffer
(progn
(set-buffer "X.html")
(message "html:%s" my-var))
;; in java buffer I get error for a void variable
(progn
(set-buffer "X.java")
(message "java:%s" my-var))