Unicode BIDI (UBA): Test = RLE ES LRE -> Difficulties following the specs - unicode

This is quite a specific bidi algorithm question, I am hoping someone out there can help my understanding of the algorithm.
I am having a hard time with the the following test from the BidiTest-10.0.0d5.txt
#Levels x, 1, x
#Reorder 1
RLE ES LRE; 2
http://unicode.org/cldr/utility/bidi.jsp?a=%E2%80%AB%2B%E2%80%AA&p=LTR
The unicode tool above applies rule W6 on the RLE character and converting it to an ON.
Why is that? Rule W6 states:
"Otherwise, separators and terminators change to Other Neutral".
obviously I am missing some detail, I just can't seem find it.
Any hint appreciated, thank you :)

I think this is what is happening. The implementation may be designed to retain, rather than remove, boundary-neutral and explicit formatting characters (see the implementation notes). Thus rule X9 adds the action "do not remove any characters, but turn all RLE, LRE, RLO, LRO, and PDF characters into BN" and thus turns the LRE into BN, then rule W6 adds the action "change all BN types adjacent to ET, ES, or CS to ON as well" and so changes the BN that was an LRE to ON.

Related

Why using _LatexSmallFractionConstant:=1 causes bad latex to be generated?

The option _LatexSmallFractionConstant:=1 is very useful in Maple, as it makes the fraction generated in latex looks natural. Here is an example
restart;
latex(1/2);
1/2
Now using _LatexSmallFractionConstant:=1, the output becomes
restart;
_LatexSmallFractionConstant:=1:
latex(1/2);
{\frac{1}{2}}
Which is the correct Latex.
But _LatexSmallFractionConstant:=1 causes big problem in other places, like here
mu:=1/((4*t+1)^(8/5)*(t-1)^(7/5));
Now see what happens when latex is generated with _LatexSmallFractionConstant:=1 set
_LatexSmallFractionConstant:=1:
latex(simplify(mu))
{1 \left( 4\,t+1 \right) ^{-{\frac{8}{5}}} \left( t-1 \right) ^{-{
\frac{7}{5}}}}
Which is completely wrong. It renders as
Removing _LatexSmallFractionConstant:=1: gives
restart;
mu:=1/((4*t+1)^(8/5)*(t-1)^(7/5));
latex(simplify(mu))
{\frac {1}{ \left( 4\,t+1 \right) ^{8/5} \left( t-1 \right) ^{7/5}}}
Which renders correctly.
Is there a way to correct the above, so I can still use _LatexSmallFractionConstant:=1: ? I need to use this option to make fractions appear in correct Latex, but I also do not want to obtain bad Latex in other places like the above.
Is there a solution to this problem?
Maple 2018.1 on windows.
It's a bug. You should report it. I'd link to where, but at the moment, it looks like both www.maplesoft.com and www.mapleprimes.com are down.

Speeding up integration methods MATLAB

I am using MATLAB to execute a triple integral using integral3 and it is running very slow. I was wondering if there ways to speed it. I am guessing its due to the fact that I set the abstol wrong. Not sure how to handle it. PS the code below works with no syntax error. There are a couple of things I dont know how to pick, abstol, method etc..
clear all
syms gamma1
syms gamma2
syms z
syms v
Nt=16; sigmanoise=10^(-7.9); c3=0.129; c1=(1-c3)/2;a2=0;b2=0;
a1=0.0030; b1= 0.0030; A1= 1.5625e-04,A2=0; B1= 7.8125e-05;B2=0;
theta= 3.1623;lambda1= 4.9736e-05;lambda2=0;p1=1;p2=0; alpha1=2; alpha2=4;delta1=2/alpha1; delta2=2/alpha2;beta1=0.025; beta2=0.025;
a= gamma1^-1+gamma2^-1+2*gamma1^(-0.5)*gamma2^(-0.5);
laplacesgi=(exp(+2*pi*j.*z*a)-1)./(2*pi*j*z);
laplacesgi=matlabFunction(laplacesgi);
laplacenoi=exp(-2*pi*j.*z*theta*sigmanoise/Nt);
laplacenoi=matlabFunction(laplacenoi);
interfere= #(gamma1,gamma2,v,z)( (1 -2*c1-c3./(1+2*pi*j*z*theta*v.^(-1))).*(A1.*(v).^(delta1-1).*exp(-a1.*(v).^ (delta1./2))+B1.*(v).^(delta2-1) .*(1-exp(-b1.*(v).^ (delta2./2)))));
gscalar =#(gamma1,gamma2,z)integral(#(v)(interfere(gamma1,gamma2,v,z)),gamma2,inf);
g = #(gamma1,gamma2,z)arrayfun(gscalar,gamma1,gamma2,z);
lp= A1*(gamma1)^(delta1-1)*exp(-a1*(gamma1)^ (delta1/2))+B1*(gamma1)^(delta2-1)*(1-exp(-b1*(gamma1)^ (delta2/2)))+A2*gamma1^(delta1-1)*exp(-a2*gamma1^(delta1/2))+ B2*gamma1^(delta2-1)*(1-exp(-b2*gamma1^ (delta2/2)));%;
dk1=((2*pi*lambda1))/(beta1^2)*(1-exp(-a1*(gamma2)^(delta1/2))*(1+(gamma2)^(delta1/2)*a1))+ pi*lambda1*gamma2^(delta2)*p1^delta2-((2*pi*lambda1)/(beta1^2))*(1-exp(-b1*(gamma2)^(delta2/2))*(1+(gamma2)^(delta2/2)*b1));
dk2=((2*pi*lambda2))/(beta2^2)*(1-exp(-a2*(gamma2)^(delta1/2))*(1+(gamma2)^(delta1/2)*a2))+ pi*lambda2*gamma2^(delta2)*p2^delta2-((2*pi*lambda2)/(beta2^2))*(1-exp(-b2*(gamma2)^(delta2/2))*(1+(gamma2)^(delta2/2)*b2));
dk=dk1+dk2;
lcp= A1*(gamma2)^(delta1-1)*exp(-a1*(gamma2)^ (delta1/2))+B1*(gamma2)^(delta2-1)*(1-exp(-b1*(gamma2)^ (delta2/2)))+A2*gamma2^(delta1-1)*exp(-a2*gamma2^ (delta1/2))+ B2*gamma2^(delta2-1)*(1-exp(-b2*gamma2^(delta2/2)));%;
pdflast=lp*lcp*exp(-dk);
pdflast=matlabFunction(pdflast);
pdflast= #(gamma1,gamma2)arrayfun(pdflast,gamma1,gamma2);
gamma2min=#(gamma1)gamma1;
warning('off','MATLAB:integral:MinStepSize');
T = integral3(#(gamma1,gamma2,z)(laplacenoi(z).*laplacesgi(gamma1,gamma2,z).*pdflast(gamma1,gamma2).*exp(-g(gamma1,gamma2,z))),0,inf,#(gamma2)gamma2,inf,0.05,1000,'abstol',1e-3)
I appreciate any ideas or suggestions.
This is getting way too long for a comment, and while it doesn't really give an answer either, I think it may be helpful anyway, so I will slightly abuse the answer form for it.
Code Readability
I don't think your code as it stands fulfills the basic fundamental purpose of code: Communicating with a human being, probably yourself down the road.
I don't know if the variable names are unambiguous enough that in six months, they will still tell you exactly what is what. If they are, great. If not, you may want to improve upon them. (And yes, naming stuff is one of the hardest parts of programming, but that doesn't make it less important.)
The same holds true for comments: If you don't need comments on your formulas, more power to you. I have no idea what you are computing, so the fact that I don't understand your formulas doesn't mean much. But again, think of yourself in a few months, looking for a problem: Would you have wished for a comment such that you know if that factor is really correct or off by one?
Here's something I do know: Your formulas are simply too wide to be comprehended at once. Simple reformatting helps to see the structure better. Here's how I reformatted your code to start making heads or tails from it:
clear all
syms gamma1
syms gamma2
syms z
syms v
Nt=16;
sigmanoise=10^(-7.9);
c3=0.129;
c1=(1-c3)/2;
a2=0;
b2=0;
a1=0.0030;
b1=0.0030;
A1=1.5625e-04;
A2=0;
B1=7.8125e-05;
B2=0;
theta=3.1623;
lambda1=4.9736e-05;
lambda2=0;
p1=1;
p2=0;
alpha1=2;
alpha2=4;
delta1=2/alpha1;
delta2=2/alpha2;
beta1=0.025;
beta2=0.025;
a=gamma1^(-1)+gamma2^(-1)+2*gamma1^(-0.5)*gamma2^(-0.5);
laplacesgi=matlabFunction((exp(2*pi*1j*z*a)-1)./(2*pi*1j*z));
laplacenoi=matlabFunction(exp(-2*pi*1j*z*theta*sigmanoise/Nt));
interfere= #(gamma1,gamma2,v,z)( ...
(1 -2*c1-c3./(1+2*pi*j*z*theta*v.^(-1))).*(A1.*v.^(delta1-1).* ...
exp(-a1.*v.^(delta1./2))+B1.*v.^(delta2-1).*(1-exp(-b1.*v.^(delta2./2)))));
gscalar=#(gamma1,gamma2,z)integral(#(v)(interfere(gamma1,gamma2,v,z)),gamma2,inf);
g=#(gamma1,gamma2,z)arrayfun(gscalar,gamma1,gamma2,z);
lp=A1*gamma1^(delta1-1)*exp(-a1*gamma1^(delta1/2))+ ...
B1*gamma1^(delta2-1)*(1-exp(-b1*gamma1^(delta2/2)))+ ...
A2*gamma1^(delta1-1)*exp(-a2*gamma1^(delta1/2))+ ...
B2*gamma1^(delta2-1)*(1-exp(-b2*gamma1^(delta2/2)));
dk1=((2*pi*lambda1))/(beta1^2)*(1-exp(-a1*gamma2^(delta1/2))*(1+gamma2^(delta1/2)*a1))+ ...
pi*lambda1*gamma2^(delta2)*p1^delta2- ...
((2*pi*lambda1)/(beta1^2))*(1-exp(-b1*gamma2^(delta2/2))*(1+gamma2^(delta2/2)*b1));
dk2=((2*pi*lambda2))/(beta2^2)*(1-exp(-a2*gamma2^(delta1/2))*(1+gamma2^(delta1/2)*a2))+ ...
pi*lambda2*gamma2^(delta2)*p2^delta2- ...
((2*pi*lambda2)/(beta2^2))*(1-exp(-b2*gamma2^(delta2/2))*(1+gamma2^(delta2/2)*b2));
dk=dk1+dk2;
lcp=A1*gamma2^(delta1-1)*exp(-a1*gamma2^(delta1/2))+ ...
B1*gamma2^(delta2-1)*(1-exp(-b1*gamma2^(delta2/2)))+ ...
A2*gamma2^(delta1-1)*exp(-a2*gamma2^(delta1/2))+ ...
B2*gamma2^(delta2-1)*(1-exp(-b2*gamma2^(delta2/2)));
pdflast=matlabFunction(lp*lcp*exp(-dk));
pdflast=#(gamma1,gamma2)arrayfun(pdflast,gamma1,gamma2);
gamma2min=#(gamma1)gamma1;
T = integral3(#(gamma1,gamma2,z)( ...
laplacenoi(z).*laplacesgi(gamma1,gamma2,z).*pdflast(gamma1,gamma2).*exp(-g(gamma1,gamma2,z))), ...
0,inf,...
#(gamma2)gamma2,inf,...
0.05,1000,...
'abstol',1e-3)
A few notes one this:
MATLAB is one of the languages that require an indication that the logical line should continue after the physical line break. The indication in MATLAB is three dots.
Get rid of any and all warnings the MATLAB editor shows you. In very rare cases, by disabling the warning for this line; usually, by correcting your code. Some of these warnings may seem over-protective, but coe quickly reaches the point where none of us can have enough of it in our minds to see the more subtle problems, and linting helps avoid a fair number of them, in my experience.
Consistent spacing helps, in the same way “proper” (i.e., standardized) spelling makes reading English easier: The patterns are just much more obvious.
Line breaks should in general not be done haphazardly, but emphasizing the structure of commands and formulas. In several of your formulas, I have seen symmetries between input parameters and tried to make them obvious by placing the line breaks accordingly. That helps a lot when looking for typos.
Your code has lines such as these:
pdflast=lp*lcp*exp(-dk);
pdflast=matlabFunction(pdflast);
I used to recycle variables like that, too. Over time, I learned the hard way that it helps for debugging and readability not to, especially if your values have different types, as they do here.
There are a few points I would still clean up at this point. For example, pdflast works just fine on arrays and the line pdflast= #(gamma1,gamma2)arrayfun(pdflast,gamma1,gamma2); should be deleted, and the lower bound for gamma2 in the integral3 call is a function of gamma1 and should be changed to #(gamma1)gamma1.
Does the computer/MATLAB care about any of this? Maybe something slipped in where it does, but basically: No. All of these changes are for you, and if you send your code in an SO post, for us, the readers.
(Likely) Bug: Vectorization
I think your definition of g is wrong:
g=#(gamma1,gamma2,z)arrayfun(gscalar,gamma1,gamma2,z);
The cubature (i.e., integral3) will try to call this function with non-scalar values for one or more of the parameters. Most likely, these will not all be of the same size, and even if they were, it would expect to get a 3D result, not a vector. Try calling your g that way:
>> g(1:2,1,1)
Error using arrayfun
All of the input arguments must be of the same size and shape.
Previous inputs had size 2 in dimension 2. Input #3 has size 1
Error in #(gamma1,gamma2,z)arrayfun(gscalar,gamma1,gamma2,z)
It's really a good idea to check intermediate building blocks like that. What your really need to have is an arrayfun over gamma2, something like this:
gscalar=#(gamma1,gamma2,z) ...
integral(#(v)(interfere(gamma1,gamma2,v,z)),gamma2,inf, ...
'ArrayValued',true);
g = #(gamma1,gamma2,z)arrayfun(#(gamma2)gscalar(gamma1,gamma2,z),gamma2);
(Possible) Bug: Definition of interfere
I don't know if you tried checking interfere against any known or suspected values. (Sanity checks for formulas I just typed seem a really good idea to me.) I somehow doubt that the formula is correctly capturing your intent:
interfere=#(gamma1,gamma2,v,z)( ...
(1-2*c1-c3./(1+2*pi*1j*z*theta*v.^(-1))).*(A1.*v.^(delta1-1).* ...
exp(-a1.*v.^(delta1./2))+B1.*v.^(delta2-1).*(1-exp(-b1.*v.^(delta2./2)))));
The potential problem with this formula (apart from a somewhat inconsistent use of * vs. .* etc.) is that the values do not depend on gamma1 and gamma2 at all.
Of course, that can happen, but if you actually mean it to be the case, what is the rationale for including gamma1 in the formula in the first place?
If this is as it should be, you may need to still make the result the proper size: Right now, interfere simply ignores its first two inputs, which may trip up the integrator: interfere(1:3,1,1,1) should return a 3-element vector.
Concluding Thoughts
As you may have noticed, your question did not get a satisfying answer yet. Nor do I think in its current form it will. To get volunteers to look at your problem, you need to make it easy to understand what you are doing:
Start by simplifying your formulas. They may not be of interest to you anymore, but right now, they're just clutter.
Trim down your parameters. That is somehow part of the above.
Throw out things that are probably irrelevant. Apart from the point that you don't need (and probably don't want) an additional arrayfun around the matlabFunction results, symbolic math is likely to be irrelevant to your actua question on integral3. If you can ask your question without it, it may attract more attention.
For anything you cannot trim down, consider explaining what is happening.
Of course, in this process, for each iteration, test your code (after saying clear all or in a fresh MATLAB session!) to check if the problem is still there. If it is not, you may have found a hint where your basic problem is hiding.
For a longer discussion on the topic, see https://meta.stackexchange.com/questions/18584/how-to-ask-a-smart-question and the guides linked to within that discussion.

Data interpretation between Matlab and Fortran

I have some Fortran code that interprets a binary file. Thanks to another question I asked I understand how the fortran code works but I need help changing it to MATLAB code. Here is the Fortran code:
IMPLICIT NONE
DOUBLE PRECISION SCALE
CHARACTER*72 BLINE
CHARACTER*72 DATA_FILE_FULL_NAME
CHARACTER*6 DATA_FILE_NAME
CHARACTER*4 CH4, CH4F
REAL*4 RL4
EQUIVALENCE (RL4,CH4)
CHARACTER*2 C2
LOGICAL LFLAG
INTEGER*2 I2
if(i2.eq.13881) LFLAG=.FALSE.
c2='69'
LFLAG=.TRUE.
DATA_FILE_FULL_NAME='./'//DATA_FILE_NAME//'.DAT'
OPEN(UNIT=20, FILE=DATA_FILE_FULL_NAME, ACCESS='DIRECT',
. RECL=72, status='OLD')
READ(20,REC=1) BLINE
CH4f=BLINE(7:10)
call flip(4,lflag,ch4f,ch4)
SCALE=RL4
RETURN
END
c ..................................................
subroutine flip(n,lflag,ch1,ch2)
c ..................................................
integer*4 n, i
character*(*) ch1, ch2
logical lflag
if(lflag) then
do i=1,n
ch2(i:i)=ch1(n-i+1:n-i+1)
enddo
else
ch2=ch1
endif
return
end
So basically (and I believe I understand this correctly) the Fortran code is checking the endianess and flipping it if the machine and the data don't match. Additionally, the data is stored in memory in a place reserved for both CH4 and RL4 so by calling RL4 after defining CH4 the data is simply being cast to the REAL*4 type instead of the CHARACTER*4 type. Now I need to figure out how to port this into Matlab. I already have the capability to read in the raw data, but when I try various forms of interpreting it I always get the wrong answer. So far I have tried:
fid=fopen(LMK_file,'r');
BLINE=fread(fid, 72,'char=>char');
CH4=BLINE(7:10);
SCALE=single(CH4(1)+CH4(2)+CH4(3)+CH4(4));
SCALE=int8(CH4(1)+256*int8(CH4(2))+256^2*int8(CH4(3))+256^3*int8(CH4(4));
in MATLAB but both give me the same wrong answer (which makes sense since its doing pretty much the same thing).
Any suggestions?
For anyone else who is looking for the answer to this question this is what I came up with thanks to the help in the comments.
fid=fopen(LMK_file,'r'); %open the file for reading
IGNORE=fread(fid,6,'*char'); %ignore the first 6 bytes. This could also be performed using fseek
SCALE=fread(fid,1,'*float32','b'); %reading in the scale value as a floating point 32 bit number.
where the b indicates the Big endianess of the file. By using the '*float32' option, fread automatically interprets the next 4 characters as a 32 bit floating point number.
This returns the correct value of scale from the given file.

for loop range syntax

I spent a couple of hours debugging a problem that I would have thought would have been a syntax error.
a = zeros(3);
for i=1:1size(a,2) % note the missing colon between 1 and size(a,2)
i
end
The following only displays
ans = 3
1
Essentially, it seems Matlab/Octave parses the above as:
for i=1:1
size(a,2)
i
end
Note however that
i=1:1size(a,2)
produces a syntax error. Is there a good reason that Matlab/Octave has this for loop syntax? Is there something that it's supposed to make easier? Just curious if anyone else has any thoughts about it. Thanks.
It is indeed a bit of a surprise that Matlab's syntax allows this. I don't know why this is allowed. One reason might be to allow for-loops on one line:
>> for i=1:3 disp(i);end
1
2
3
But interestingly, removing the space is not allowed:
>> for i=1:3disp(i);end
for i=1:3disp(i);end
|
Error: Unexpected MATLAB operator.
This reason for this is probably that a number followed by d is another way of writing a floating point number (3d10 == 3e10), so the parser/tokenizer initially thinks you define a number, but then gets confused when it sees the i. Daniel's example with fprintf does work, since a number followed by an f is not a valid number, so the tokenizer understands that you started a new token.
I guess that many years ago (>30?), when they defined matlab's syntax, they did not foresee that this could introduce this kind of hard-to-spot problems. I guess matlab was originally written by engineers for engineers, and not by someone who knows how to design a general purpose programming language. Other languages like C or Python use punctuation to separate loop conditions from loop body, so there is no ambiguity. I don't know if it is still possible to correct Matlab's syntax, since it might break old code that relies on the current behavior.
At least, if you use a recent version of Matlab, the editor warns for various problems in your code. Paying attention to the small red dashes in the right hand border could have saved you a few hours of debugging time (but maybe you were using octave). I try to make it a habit to fix all the warnings it indicates. For your code, it shows the following:
Your code is equivalent to
a = zeros(3);
for i=1:1
size(a,2)
i
end
There are some places where everyone would use newline or white space, but the parser itself does not require.
A minimal loop:
for i=1:3fprintf('%d',i),end
but I recommend to use at least a comma seperated version, everything else is horrible to read:
for i=1:3,fprintf('%d',i),end

Why is not equal represented by ~= in Matlab?

This one of the things that has always bothered me about Matlab. I understand why arrays start at 1 and not at 0 like in any other programming language, but why is != ~= in Matlab?
The tilde character (~) is generally used as the bitwise NOT operator.
As the ! character is reserved for an other usage (OS command), I guess it's not a bad choice.
In mathematical logic ~ is an old-fashioned way to write ¬
In logic tilde can mean "not", which may be confusing as in math tilde can be "equivalence" or "approx". However, it is found on more keyboards than the less ambiguous ¬. Watch out, as tilde can also mean bitwise not :)
Why is .not..eq. represented by ~= ? For the same reason that it is not represented as =!= or /= or any of the hundred and one other conventions used in programming languages.
And the twiddle, or ~, is widely used in logic texts to mean NOT.