Method types incompatiblity - streaming

Question

I tried recently play with a streaming liquidsoap-like stuff... There's some code which uses OCaml classes and C libraries for encoding like lame (via ocaml-lame) etc.
(* Lame module *)
type encoder
(* ... *)
external encode_buffer_float_part : encoder -> float array -> float array -> int -> int -> string = "ocaml_lame_encode_buffer_float"
(* Otherencoder module *)
type encoder
(* ... *)
external encode_buffer_float_part : encoder -> float array -> float array -> int -> int -> string = "ocaml_otherencoder_encode_buffer_float"
(=same interface)
Somewhere there's a two high-level classes that inherit from two separated encoderbase virtual classes:
(* Mp3_output module *)
class virtual encoderbase =
object (self)
method encode ncoder channels buf offset size =
if channels = 1 then
Lame.encode_buffer_float_part ncoder buf.(0) buf.(0) offset size
else
Lame.encode_buffer_float_part ncoder buf.(0) buf.(1) offset size
end
(* somewhere in the code *)
class to_shout sprop =
(* some let-s *)
object (self)
inherit
[Lame.encoder] Icecast2.output ~format:Format_mp3 (* more params *) as super
inherit base
(* ... *)
end
and
(* Other_output module *)
class virtual encoderbase =
object (self)
method encode ncoder channels buf offset size =
if channels = 1 then
Otherencoder.encode_buffer_float_part ncoder buf.(0) buf.(0) offset size
else
Otherencoder.encode_buffer_float_part ncoder buf.(0) buf.(1) offset size
end
(* somewhere in the code *)
class to_shout sprop =
(* some let-s *)
object (self)
inherit
[Otherencoder.encoder] Icecast2.output ~format:Format_other (* more params *) as super
inherit base
(* ... *)
end
All things work fine with:
let icecast_out source format =
let sprop =
new Mp3_output.shout_sprop
in
(* some code here *)
new Mp3_output.to_shout sprop
but when I try something like this:
let icecast_out source format =
let sprop =
if format = Format_other then
new Other_output.shout_sprop
else
new Mp3_output.shout_sprop
in
(* some code here *)
if format = Format_mp3 then
new Mp3_output.to_shout sprop
else
new Other_output.to_shout sprop
compilation breaks with an error # new Other_output.to_shout sprop:
Error: This expression has type Other_output.to_shout
but an expression was expected of type Mp3_output.to_shout
Types for method encode are incompatible
Is there any way to "convince" OCaml (common ancestor? wrapping class? type casting?) to compile with that two different classes/bindings at once?
Update (2015.12.15):
Code sample: https://gist.github.com/soutys/22b67a5df9ae0a6f1f72

Answer 1

Is there any way to "convince" OCaml (common ancestor? wrapping class? type casting?) to compile with that two different classes/bindings at once?
It is believed that OCaml is a type safe language, so it is not possible to convince OCaml to compile a program that will crash, unless you're using some sinister methods.
The root of your misunderstanding is illustrated by the following code snippet from your example:
type 'a term = Save of 'a
let enc_t =
if format_num = 1
then Save Lame.encoder
else Save Other.encoder
An expression Save Lame.encoder has a type Lame.encoder term, while an expression Save Other.encoder has a type Other.encoder term. From a type system perspective this are two completly different types, albeit they were built by the same type constructor term. C.f., int list and float list are different types, and you can't assign to the same variable values of this two different types. That is not a property of OCaml per se, this is a property of any parametric polymorphism, e.g., std::vector<int> and std::vector<float> are different types with different representation and their values can't be used interchangeably, although strictly speaking templates in c++ are not true parametric polymorphism, they are just macro.
But back to OCaml. The idea behind the polymorphic function is that a parameter which has a polymorphic data type has the same representation, and function can be applied to any instance of this type without any runtime check, since all type information is removed. For example,
let rec length = function
| [] -> 0
| _ :: xs -> 1 + length xs
is polymorphic because it can be applied to any instance of polymorphic data type 'a list, including int list, float list, person list, etc.
On the other hand, a function
let rec sum = function
| [] -> 0
| x :: xs -> x + sum xs
is not polymorphic and can be applied only to values of type int list. The reason is because the implementation of this function relies on a fact, that every element is an integer. If you will be able to convince the type system, to apply this function to float list, you will get a segmentation fault.
But you may say, that we miss something, as a sum function for float list looks basically the same:
let rec fsum = function
| [] -> 0.
| x :: xs -> x +. fsum xs
So there is an opportunity to abstract a summation. When we abstract something we find the things that differ between different implementations and abstract them out. The simplest abstraction primitive in OCaml is a function, so lets do it:
let rec gsum zero plus xs =
let (+) = plus in
let rec sum = function
| [] -> zero
| x :: xs -> x + sum xs in
sum xs
We abstracted out the zero element and a plus function. So that we get an abstraction of summation, that works for any type for which you can provide a plus operation and element neutral to this opperation (called a Ring data stucture in abstract algebra). The type of the gsum is
'a -> ('b -> 'a -> 'a) -> 'b list -> 'a
It is even too generic, and we can specialize it a little bit, as this type
'a -> ('a -> 'a -> 'a) -> 'a list -> 'a
suits better. Instead of passing elements of the ring structure one by one, we can aggregate it into a record type:
type 'a ring = {
zero : 'a;
plus : 'a -> 'a -> 'a
}
and implement our generic sum as follows:
let rec gsum ring xs =
let (+) = ring.plus in
let rec sum = function
| [] -> ring.zero
| x :: xs -> x + sum xs in
sum xs
In that case we will have a nice type sum : 'a ring -> 'a list -> 'a. At some point of time, you will find yourself extending this record with new fields, and implementing more and more functions, that accepts this ring structure as a first parameter. And this would be a good time to use a more heavy abstraction, called functor. The functor is actually a record on steroids, that is implicitly passed to each function of the functor implementation. Other then functions, records of functions and functors there're other abstraction techniques: first-class modules, objects, classes and, soon, implicits (aka type classes). It is a part of the programming expertise to learn how to choose an abstraction technique that suits better in each particular case. The general advice would be to use least heavy one if possible. And indeed in 95% using functions or records of functions is enough.
Now let's go back to your example. Here you're hitting in the same hole,
you're confusing polymorpism with abstraction:
let fmt =
if format_num = 1
then new Lamefmt.sout sp
else new Otherfmt.sout sp in
Lamefmt.sout and Otherfmt.sout are different types, the first one has type:
type sout = <
doenc : Lame.encoder -> float array array -> int -> int -> string;
encode : Lame.encoder -> float array array -> int -> int -> string
>
and the second:
type sout = <
doenc : Other.encoder -> float array array -> int -> int -> string;
encode : Other.encoder -> float array array -> int -> int -> string
>
This are two different objects, although they have resembling scheme, and that means that we have some abstraction opportunity here.
In your case we can start with a simple observation, that both encoder functions has the same type modulo the encoder object itself. Using Occam's razor principle, we're trying to catch this with the simplest abstraction possible -- a function:
type encoder = buffer -> buffer -> int -> int -> string
where
type buffer = float array array
Then we can construct different encoders:
let lame : encoder =
let encoder = Lame.create_encoder () in
Lame.encode_buffer_float_part encoder
let other : encoder =
let encoder = Other.create_encoder () in
Other.encode_buffer_float_part encoder
And then you can use this two values interchangaebly. Sometimes, different encoders will require different parameters, in that case our task is to dispatch them as soon as possible, e.g.,
let very_customizable_encoder x y z : encoder =
let encoder = VCE.create_encoder x y z in
Other.encode_buffer_float_part encoder
In that case, you should resolve the customization issue as close to user as possible, and later work with the abstraction.
It is quite often to use hashtables or other associative data structures to store encoders. Such approach will even allow you to represent a plugin architecture, where plugins are loaded dynamically and register themselves (the value of type encoder) in some table.
Conclusion. It is enough to use a function to represent your problem. Maybe at some point of time you will need to use records of functions. So far I don't see the need for using classes. Usually, they are required when you're interested in open recursion, i.e., when your problem is represented by a set of mutual recursive functions, and you want to leave some of the functions unspecified, i.e., to parametrize the implementation.

Answer 2

The first thing is to simplify your example. e.g. removing irrelevant bits and adding dummy stubs so we don't need real encoders:
module Lame = struct
type encoder
let encode_buffer_float_part : encoder -> float -> unit = fun _ -> failwith "ocaml_lame_encode_buffer_float"
end
module Otherencoder = struct
type encoder
let encode_buffer_float_part : encoder -> float -> unit = fun _ -> failwith "ocaml_otherencoder_encode_buffer_float"
end
module Mp3_output = struct
class to_shout = object
method encode ncoder x =
Lame.encode_buffer_float_part ncoder x
end
end
module Other_output = struct
class to_shout = object
method encode ncoder x =
Otherencoder.encode_buffer_float_part ncoder x
end
end
type format = Format_other | Format_mp3
let icecast_out source format =
if format = Format_mp3 then new Mp3_output.to_shout
else new Other_output.to_shout
gives
Error: This expression has type Other_output.to_shout
but an expression was expected of type Mp3_output.to_shout
Types for method encode are incompatible
Which is correct. If I give you "a value returned by icecast_out" then you wouldn't know how to call it, because you wouldn't know what the first argument should be.
Since the encoder is different for each class, it should probably be a constructor argument. e.g.
module Mp3_output = struct
class to_shout ncoder = object
method encode x =
Lame.encode_buffer_float_part ncoder x
end
end
module Other_output = struct
class to_shout ncoder = object
method encode x =
Otherencoder.encode_buffer_float_part ncoder x
end
end
type format = Format_other | Format_mp3
let icecast_out source format =
if format = Format_mp3 then new Mp3_output.to_shout Lame.ncoder
else new Other_output.to_shout Otherencoder.ncoder