Ordering a list of lists subject to constraints - scala

Question

I have encountered a surprisingly challenging problem arranging a matrix-like (List of Lists) of values subject to the following constraints (or deciding it is not possible):
A matrix of m randomly generated rows with up to n distinct values (no repeats within the row) arrange the matrix such that the following holds (if possible):
1) The matrix must be "lower triangular"; the rows must be ordered in ascending lengths so the only "gaps" are in the top right corner
2) If a value appears in more than one row it must be in the same column (i.e. rearranging the order of values in a row is allowed).
Expression of the problem/solution in a functional language (e.g. Scala) is desirable.
Example 1 - which has a solution
A B
C E D
C A B
becomes (as one solution)
A B
E D C
A B C
since A, B and C all appear in columns 1, 2 and 3, respectively.
Example 2 - which has no solution
A B C
A B D
B C D
has no solution since the constraints require the third row to have the C and D in the third
column which is not possible.

Answer 1

I thought this was an interesting problem and have modeled a proof-of-concept-version in MiniZinc (a very high level Constraint Programming system) which seems to be correct. I'm not sure if it's of any use, and to be honest I'm not sure if it's powerful for very largest problem instances.
The first problem instance has - according to this model - 4 solutions:
B A _
E D C
B A C
----------
B A _
D E C
B A C
----------
A B _
E D C
A B C
----------
A B _
D E C
A B C
The second example is considered unsatisfiable (as it should).
The complete model is here: http://www.hakank.org/minizinc/ordering_a_list_of_lists.mzn
The basic approach is to use matrices, where shorter rows are filled with a null value (here 0, zero). The problem instance is the matrix "matrix"; the resulting solution is in the matrix "x" (the decision variables, as integers which are then translated to strings in the output). Then there is a helper matrix, "perms" which are used to ensure that each row in "x" is a permutation of the corresponding row in "matrix", done with the predicate "permutation3". There are some other helper arrays/sets which simplifies the constraints.
The main MiniZinc model (sans output) is show below.
Here are some comments/assumptions which might make the model useless:
this is just a proof-of-concept model since I thought it was an interesting
problem.
I assume that the rows in the matrix (the problem data) is already ordered
by size (lower triangular). This should be easy to do as a preprocessing step
where Constraint Programming is not needed.
the shorter lists are filled with 0 (zero) so we can work with matrices.
since MiniZinc is a strongly typed language and don't support
symbols, we just define integers 1..5 to represent the letters A..E.
Working with integers is also beneficial when using traditional
Constraint Programming systems.
% The MiniZinc model (sans output)
include "globals.mzn";
int: rows = 3;
int: cols = 3;
int: A = 1;
int: B = 2;
int: C = 3;
int: D = 4;
int: E = 5;
int: max_int = E;
array[0..max_int] of string: str = array1d(0..max_int, ["_", "A","B","C","D","E"]);
% problem A (satifiable)
array[1..rows, 1..cols] of int: matrix =
array2d(1..rows, 1..cols,
[
A,B,0, % fill this shorter array with "0"
E,D,C,
A,B,C,
]);
% the valid values (we skip 0, zero)
set of int: values = {A,B,C,D,E};
% identify which rows a specific values are.
% E.g. for problem A:
% value_rows: [{1, 3}, {1, 3}, 2..3, 2..2, 2..2]
array[1..max_int] of set of int: value_rows =
[ {i | i in 1..rows, j in 1..cols where matrix[i,j] = v} | v in values];
% decision variables
% The resulting matrix
array[1..rows, 1..cols] of var 0..max_int: x;
% the permutations from matrix to x
array[1..rows, 1..cols] of var 0..max_int: perms;
%
% permutation3(a,p,b)
%
% get the permutation from a b using the permutation p.
%
predicate permutation3(array[int] of var int: a,
array[int] of var int: p,
array[int] of var int: b) =
forall(i in index_set(a)) (
b[i] = a[p[i]]
)
;
solve satisfy;
constraint
forall(i in 1..rows) (
% ensure unicity of the values in the rows in x and perms (except for 0)
alldifferent_except_0([x[i,j] | j in 1..cols]) /\
alldifferent_except_0([perms[i,j] | j in 1..cols]) /\
permutation3([matrix[i,j] | j in 1..cols], [perms[i,j] | j in 1..cols], [x[i,j] | j in 1..cols])
)
/\ % zeros in x are where there zeros are in matrix
forall(i in 1..rows, j in 1..cols) (
if matrix[i,j] = 0 then
x[i,j] = 0
else
true
endif
)
/\ % ensure that same values are in the same column:
% - for each of the values
% - ensure that it is positioned in one column c
forall(k in 1..max_int where k in values) (
exists(j in 1..cols) (
forall(i in value_rows[k]) (
x[i,j] = k
)
)
)
;
% the output
% ...

Answer 2

I needed a solution in a functional language (XQuery) so I implemented this first in Scala due to its expressiveness and I post the code below. It uses a brute-force, breadth first style search for solutions. I'm inly interested in a single solution (if one exists) so the algorithm throws away the extra solutions.
def order[T](listOfLists: List[List[T]]): List[List[T]] = {
def isConsistent(list: List[T], listOfLists: List[List[T]]) = {
def isSafe(list1: List[T], list2: List[T]) =
(for (i <- list1.indices; j <- list2.indices) yield
if (list1(i) == list2(j)) i == j else true
).forall(_ == true)
(for (row <- listOfLists) yield isSafe(list, row)).forall(_ == true)
}
def solve(fixed: List[List[T]], remaining: List[List[T]]): List[List[T]] =
if (remaining.isEmpty)
fixed // Solution found so return it
else
(for {
permutation <- remaining.head.permutations.toList
if isConsistent(permutation, fixed)
ordered = solve(permutation :: fixed, remaining.tail)
if !ordered.isEmpty
} yield ordered) match {
case solution1 :: otherSolutions => // There are one or more solutions so just return one
solution1
case Nil => // There are no solutions
Nil
}
// Ensure each list has unique items (i.e. no dups within the list)
require (listOfLists.forall(list => list == list.distinct))
/*
* The only optimisations applied to an otherwise full walk through all solutions is to sort the list of list so that the lengths
* of the lists are increasing in length and then starting the ordering with the first row fixed i.e. there is one degree of freedom
* in selecting the first row; by having the shortest row first and fixing it we both guarantee that we aren't disabling a solution from being
* found (i.e. by violating the "lower triangular" requirement) and can also avoid searching through the permutations of the first row since
* these would just result in additional (essentially duplicate except for ordering differences) solutions.
*/
//solve(Nil, listOfLists).reverse // This is the unoptimised version
val sorted = listOfLists.sortWith((a, b) => a.length < b.length)
solve(List(sorted.head), sorted.tail).reverse
}