I have two symbolic expressions a and b, each consists of polynomials with basic arithmetic and small, positive, integer powers.
simplify(a - b) doesn't go up to 0, and my only alternative is to subs some random numbers into the variables and compare.
I would have expected something like expanding the expressions until there are no parentheses. Then, add all fractions into a single fraction.
I converted the data into a function which can be called as:
x = sym('x', [1 8], 'real')';
err = func( x ) % should be simplified to zeros
x0 = rand( size(x) )
double( subs(err, x, x0) )
simplify(err)
The function
function err_Dpsi_Dpsi2 = func(in1)
%FUNC
% ERR_DPSI_DPSI2 = FUNC(IN1)
% This function was generated by the Symbolic Math Toolbox version 8.4.
% 29-Dec-2020 20:03:34
x1 = in1(1,:);
x2 = in1(2,:);
x3 = in1(3,:);
x4 = in1(4,:);
x5 = in1(5,:);
x6 = in1(6,:);
x7 = in1(7,:);
x8 = in1(8,:);
t2 = x1.*x6;
t3 = x2.*x5;
t4 = x1.*x7;
t5 = x3.*x5;
t6 = x2.*x7;
t7 = x3.*x6;
t8 = -x2;
t9 = -x3;
t10 = -x6;
t11 = -x7;
t15 = x1./2.0;
t16 = x2./2.0;
t17 = x1./4.0;
t18 = x3./2.0;
t19 = x2./4.0;
t20 = x3./4.0;
t21 = x5./2.0;
t22 = x6./2.0;
t23 = x5./4.0;
t24 = x7./2.0;
t25 = x6./4.0;
t26 = x7./4.0;
t43 = x2.*7.072e+3;
t44 = x3.*7.072e+3;
t45 = x4.*7.071e+3;
t46 = x6.*7.072e+3;
t47 = x7.*7.072e+3;
t48 = x8.*7.071e+3;
t60 = x2.*x8.*-7.071e+3;
t62 = x4.*x7.*-7.071e+3;
t69 = x1.*9.999907193999999e-1;
t70 = x5.*9.999907193999999e-1;
t71 = x1.*1.0000660704;
t72 = x5.*1.0000660704;
t74 = x2.*1.0001321408;
t75 = x3.*1.0001321408;
t76 = x6.*1.0001321408;
t77 = x7.*1.0001321408;
t78 = x1.*1.0000660704;
t79 = x2.*5.000660704e-1;
t80 = x2.*1.0001321408;
t81 = x3.*5.000660704e-1;
t82 = x3.*1.0001321408;
t83 = x5.*1.0000660704;
t84 = x6.*5.000660704e-1;
t85 = x6.*1.0001321408;
t86 = x7.*5.000660704e-1;
t87 = x7.*1.0001321408;
t102 = x1.*9.999907194000001e-1;
t103 = x5.*9.999907194000001e-1;
t104 = x4.*4.999953597e-1;
t105 = x8.*4.999953597e-1;
t108 = x2.*1.000132149530596;
t109 = x3.*1.000132149530596;
t110 = x6.*1.000132149530596;
t111 = x7.*1.000132149530596;
t112 = x2.*1.000056789186827;
t113 = x3.*1.000056789186827;
t114 = x6.*1.000056789186827;
t115 = x7.*1.000056789186827;
t124 = x4.*1.000056789186827;
t125 = x8.*1.000056789186827;
t126 = x4.*9.999814388861295e-1;
t127 = x8.*9.999814388861295e-1;
t128 = x2.*1.000132149530596;
t129 = x3.*1.000132149530596;
t130 = x6.*1.000132149530596;
t131 = x7.*1.000132149530596;
t139 = x4.*2.500307147434136e-1;
t140 = x8.*2.500307147434136e-1;
t141 = x2.*1.000056789186827;
t142 = x3.*1.000056789186827;
t144 = x4.*1.000056789186827;
t145 = x6.*1.000056789186827;
t146 = x7.*1.000056789186827;
t148 = x8.*1.000056789186827;
t157 = x2.*x8.*(-2.500307147434136e-1);
t158 = x4.*x7.*(-2.500307147434136e-1);
t159 = x4.*9.999814388861297e-1;
t160 = x8.*9.999814388861297e-1;
t12 = -t3;
t13 = -t4;
t14 = -t7;
t27 = t2./4.0;
t28 = t3./4.0;
t29 = t4./4.0;
t30 = t5./4.0;
t31 = t6./4.0;
t32 = t7./4.0;
t33 = t8+x1;
t34 = t9+x1;
t35 = t10+x5;
t36 = t11+x5;
t37 = -t16;
t38 = -t18;
t39 = -t20;
t40 = -t22;
t41 = -t24;
t42 = -t26;
t52 = t6.*7.072e+3;
t53 = t48.*x2;
t54 = t7.*7.072e+3;
t55 = t45.*x6;
t56 = t48.*x3;
t57 = t45.*x7;
t58 = -t45;
t59 = -t48;
t88 = -t74;
t89 = -t75;
t90 = -t76;
t91 = -t77;
t92 = -t80;
t93 = -t79;
t94 = -t82;
t95 = -t81;
t96 = -t85;
t97 = -t84;
t98 = -t87;
t99 = -t86;
t116 = -t108;
t117 = -t109;
t118 = -t110;
t119 = -t111;
t120 = -t112;
t121 = -t113;
t122 = -t114;
t123 = -t115;
t132 = -t128;
t133 = -t129;
t134 = -t130;
t135 = -t131;
t136 = t6.*2.500660747652978e-1;
t137 = t7.*2.500660747652978e-1;
t143 = -t139;
t147 = -t140;
t149 = t140.*x2;
t150 = t139.*x6;
t151 = t140.*x3;
t152 = t139.*x7;
t153 = -t141;
t154 = -t142;
t155 = -t145;
t156 = -t146;
t49 = -t28;
t50 = -t29;
t51 = -t32;
t61 = -t54;
t63 = t43+t58;
t64 = t44+t58;
t65 = t46+t59;
t66 = t47+t59;
t67 = t2+t5+t6+t12+t13+t14;
t138 = -t137;
t161 = t15+t38+t93+t104;
t162 = t15+t37+t95+t104;
t163 = t21+t41+t97+t105;
t164 = t21+t40+t99+t105;
t169 = t71+t89+t116+t124;
t170 = t71+t88+t117+t124;
t171 = t72+t91+t118+t125;
t172 = t72+t90+t119+t125;
t173 = t78+t92+t133+t144;
t174 = t78+t94+t132+t144;
t175 = t83+t96+t135+t148;
t176 = t83+t98+t134+t148;
t177 = t69+t120+t121+t126;
t178 = t70+t122+t123+t127;
t179 = t102+t153+t154+t159;
t180 = t103+t155+t156+t160;
t68 = 1.0./t67;
t73 = t27+t30+t31+t49+t50+t51;
t106 = t52+t55+t56+t60+t61+t62;
t165 = t161.^2;
t166 = t162.^2;
t167 = t163.^2;
t168 = t164.^2;
t182 = t136+t138+t150+t151+t157+t158;
t100 = 1.0./t73;
t107 = 1.0./t106;
t181 = t165+t166+t167+t168;
t183 = 1.0./t182;
t101 = t100.^2;
t184 = t183.^2;
err_Dpsi_Dpsi2 = [t181.*(t35.*t68.*t100-t36.*t68.*t100)+t101.*t181.*(t25+t42),-t100.*t169+t100.*t174+t169.*t183-t174.*t183-t101.*t181.*(t23+t42)+t181.*t184.*(t140-x7.*2.500660747652978e-1)+t36.*t68.*t100.*t181+t66.*t107.*t181.*t183.*1.0,-t100.*t170+t100.*t173+t170.*t183-t173.*t183-t181.*t184.*(t140-x6.*2.500660747652978e-1)+t101.*t181.*(t23-t25)-t35.*t68.*t100.*t181-t65.*t107.*t181.*t183.*1.0,t100.*t177-t100.*t179-t177.*t183+t179.*t183+t181.*(t65.*t107.*t183.*9.998585972850678e-1-t66.*t107.*t183.*9.998585972850678e-1)-t181.*t184.*(x6.*2.500307147434136e-1-x7.*2.500307147434136e-1),-t181.*(t33.*t68.*t100-t34.*t68.*t100)-t101.*t181.*(t19+t39),-t100.*t171+t100.*t176+t171.*t183-t176.*t183+t101.*t181.*(t17+t39)-t181.*t184.*(t139-x3.*2.500660747652978e-1)-t34.*t68.*t100.*t181-t64.*t107.*t181.*t183.*1.0,-t100.*t172+t100.*t175+t172.*t183-t175.*t183+t181.*t184.*(t139-x2.*2.500660747652978e-1)-t101.*t181.*(t17-t19)+t33.*t68.*t100.*t181+t63.*t107.*t181.*t183.*1.0,t100.*t178-t100.*t180-t178.*t183+t180.*t183-t181.*(t63.*t107.*t183.*9.998585972850678e-1-t64.*t107.*t183.*9.998585972850678e-1)+t181.*t184.*(x2.*2.500307147434136e-1-x3.*2.500307147434136e-1)];
I found what I was looking for
num = numden( err ) % convert to rational polynomial, and we care only about the numerator
collect( num ) % cancel terms--not needed, numden does some version of simplify
My example, though, wasn't good. For some reason, there are precision issues. I thought that symbolic used exact arithmetic, but I didn't look into that. However, if I use variables instead of finite precision coefficients, then it outputs zeros.
Related
On checking for condition gap(high-low) > 0.1%(which is met multiple times), the label only gets
rendered ONCE (instead of on relevant bars within 25 bar lookback).
Plz provide a solution.
CODE :
Historical Bars
//#version=5
indicator("PriceMomemtum",overlay = true,max_bars_back = 25)
gap = (math.abs(high - low)/low ) * 100
//var gap = (math.abs(high - low)/low ) * 100
if gap > 0.1
var lbl = label.new(x = bar_index,y = na , text = na ,text_font_family = font.family_default ,xloc = xloc.bar_index,yloc =yloc.abovebar,style = label.style_arrowdown ,textcolor = color.white,size =size.small,textalign = text.align_left,tooltip = na)
label.set_text(lbl,str.tostring(gap,"#.00")+"%")
label.set_xy(lbl,bar_index,high )
Realtime Bars
//#version=5
indicator("PriceMomemtum",overlay = true,max_bars_back = 25)
if barstate.isrealtime
gap = (math.abs(high - low)/low ) * 100
//var gap = (math.abs(high - low)/low ) * 100
if gap > 0.1
var lbl = label.new(x = bar_index,y = na , text = na ,text_font_family = font.family_default ,xloc = xloc.bar_index,yloc =yloc.abovebar,style = label.style_arrowdown ,textcolor = color.white,size =size.small,textalign = text.align_left,tooltip = na)
label.set_text(lbl,str.tostring(gap,"#.00")+"%")
label.set_xy(lbl,bar_index,high )
alert(str.tostring(time(syminfo.timezone)) + "(PriceMomentum)", alert.freq_once_per_bar)
Have you tried defining "lbl" variable without "var"?
result
I have a training function, in which inside there are two vectors:
d_labels_a = torch.zeros(128)
d_labels_b = torch.ones(128)
Then I have these features:
# Compute output
features_a = nets[0](input_a)
features_b = nets[1](input_b)
features_c = nets[2](inputs)
And then a domain classifier (nets[4]) makes predictions:
d_pred_a = torch.squeeze(nets[4](features_a))
d_pred_b = torch.squeeze(nets[4](features_b))
d_pred_a = d_pred_a.float()
d_pred_b = d_pred_b.float()
print(d_pred_a.shape)
The error raises in the loss function: ` pred_a = torch.squeeze(nets3)
pred_b = torch.squeeze(nets3)
pred_c = torch.squeeze(nets3)
loss = criterion(pred_a, labels_a) + criterion(pred_b, labels_b) + criterion(pred_c, labels) + d_criterion(d_pred_a, d_labels_a) + d_criterion(d_pred_b, d_labels_b)
The problem is that d_pred_a/b is different from d_labels_a/b, but only after a certain point. Indeed, when I print the shape of d_pred_a/b it istorch.Size([128])but then it changes totorch.Size([112])` independently.
It comes from here:
# Compute output
features_a = nets[0](input_a)
features_b = nets[1](input_b)
features_c = nets[2](inputs)
because if I print the shape of features_a is torch.Size([128, 2048]) but it changes into torch.Size([112, 2048])
nets[0] is a VGG, like this:
class VGG16(nn.Module):
def __init__(self, input_size, batch_norm=False):
super(VGG16, self).__init__()
self.in_channels,self.in_width,self.in_height = input_size
self.block_1 = VGGBlock(self.in_channels,64,batch_norm=batch_norm)
self.block_2 = VGGBlock(64, 128,batch_norm=batch_norm)
self.block_3 = VGGBlock(128, 256,batch_norm=batch_norm)
self.block_4 = VGGBlock(256,512,batch_norm=batch_norm)
#property
def input_size(self):
return self.in_channels,self.in_width,self.in_height
def forward(self, x):
x = self.block_1(x)
x = self.block_2(x)
x = self.block_3(x)
x = self.block_4(x)
# x = self.avgpool(x)
x = torch.flatten(x,1)
return x
I solved. The problem was the last batch. I used drop_last=True in the dataloader and It worked.
i want to optimize my code, and I would like to get a loop for my netcdf files. These are the files I want to open and the variables I want to use:
hydrofile2 = 'C:\modelana\netcdf_2019\westcoms2_20190402_0002.nc';
hydrofile3 = 'C:\modelana\netcdf_2019\westcoms2_20190403_0003.nc';
hydrofile4 = 'C:\modelana\netcdf_2019\westcoms2_20190404_0004.nc';
hydrofile5 = 'C:\modelana\netcdf_2019\westcoms2_20190405_0005.nc';
hydrofile6 = 'C:\modelana\netcdf_2019\westcoms2_20190406_0006.nc';
hydrofile7 = 'C:\modelana\netcdf_2019\westcoms2_20190407_0007.nc';
hydrofile8 = 'C:\modelana\netcdf_2019\westcoms2_20190408_0008.nc';
hydrofile9 = 'C:\modelana\netcdf_2019\westcoms2_20190409_0009.nc';
hydrofile10 = 'C:\modelana\netcdf_2019\westcoms2_20190410_0010.nc';
hydrofile11 = 'C:\modelana\netcdf_2019\westcoms2_20190411_0011.nc';
hydrofile12 = 'C:\modelana\netcdf_2019\westcoms2_20190412_0012.nc';
hydrofile13 = 'C:\modelana\netcdf_2019\westcoms2_20190413_0013.nc';
hydrofile14 = 'C:\modelana\netcdf_2019\westcoms2_20190414_0014.nc';
hydrofile15 = 'C:\modelana\netcdf_2019\westcoms2_20190415_0015.nc';
%velocity
u2 = ncread(hydrofile2,'u');
u3 = ncread(hydrofile3,'u');
u4 = ncread(hydrofile4,'u');
u5 = ncread(hydrofile5,'u');
u6 = ncread(hydrofile6,'u');
u7 = ncread(hydrofile7,'u');
u8 = ncread(hydrofile8,'u');
u9 = ncread(hydrofile9,'u');
u10 = ncread(hydrofile10,'u');
u11 = ncread(hydrofile11,'u');
u12 = ncread(hydrofile12,'u');
u13 = ncread(hydrofile13,'u');
u14 = ncread(hydrofile14,'u');
u15 = ncread(hydrofile15,'u');
%salinity
s2 = ncread(hydrofile2,'salinity');
s3 = ncread(hydrofile3,'salinity');
s4 = ncread(hydrofile4,'salinity');
s5 = ncread(hydrofile5,'salinity');
s6 = ncread(hydrofile6,'salinity');
s7 = ncread(hydrofile7,'salinity');
s8 = ncread(hydrofile8,'salinity');
s9 = ncread(hydrofile9,'salinity');
s10 = ncread(hydrofile10,'salinity');
s11 = ncread(hydrofile11,'salinity');
s12 = ncread(hydrofile12,'salinity');
s13 = ncread(hydrofile13,'salinity');
s14 = ncread(hydrofile14,'salinity');
s15 = ncread(hydrofile15,'salinity');
%temperature
t2 = ncread(hydrofile2,'temp');
t3 = ncread(hydrofile3,'temp');
t4 = ncread(hydrofile4,'temp');
t5 = ncread(hydrofile5,'temp');
t6 = ncread(hydrofile6,'temp');
t7 = ncread(hydrofile7,'temp');
t8 = ncread(hydrofile8,'temp');
t9 = ncread(hydrofile9,'temp');
t10 = ncread(hydrofile10,'temp');
t11 = ncread(hydrofile11,'temp');
t12 = ncread(hydrofile12,'temp');
t13 = ncread(hydrofile13,'temp');
t14 = ncread(hydrofile14,'temp');
t15 = ncread(hydrofile15,'temp');
So far, I've just been able to do this:
%% add hydrofiles
clear
myFolder = ('C:\modelana\netcdf_2019\');
if ~isfolder(myFolder)
errorMessage = sprintf('Error: The following folder does not exist:\n%s', myFolder);
uiwait(warndlg(errorMessage));
return;
end
for k = 1:14
ncFilename = sprintf('westcoms2_20190402_0002.nc',k);
if isempty(ncFilename == 1)
continue;
else
ncfile=([myFolder ncFilename]);
s = ncread(ncfile,'salinity') ;
t = ncread(ncfile,'temp') ;
u=ncread(ncfile,'u');
end
end
So I just get one .nc file open with its corresponding variables. But what I want is to be able to get all the files and the variables by using the loop.
Thanks and apologize for this basic question
You almost have it, you are missing basic string concatenation/file reading.
You can concatenate strings as [str1, str2, 'random charaters']. e.g. you can do
['westcoms2_2019040', num2str(k), '_000', num2str(k) ,'.nc']
This you can do if you have obvious file formats and some constrains. If you want to read all of them, you can do
fileList = dir([myFolder '*.nc']);
To just get all files with that extension.
Also, remember to store variables independently. i.e.
t(k,:) = ncread(ncfile,'temp') ; % if it is an array
t{k} = ncread(ncfile,'temp') ; % if it is some complex structure
i'm new to the Hadoop world and i'm having some trouble with my final data.
My purpose is to extract data from a facebook page (i'm using restfb API) using flume, then the data goes to HDFS which will be used by HIVE to gerenerate the final data. This happens every hour. All this on HUE.
I don't know why, but sometimes I success in extract data from the hole day. And some days, I can only extract data from a few hours.
This is the data from Flume:
As you can see, on 03/21 I could only extract the first 4h from the day. While on 03/22, I could extract the hole day.
Some more info.
My Flume config. from Cloudera Manager
FacebookAgent.sources = FacebookPageFansCity FacebookPageFansGenderAge FacebookPageFans FacebookPagePosts FacebookPageViews
FacebookAgent.channels = MemoryChannelFacebookPageFansCity MemoryChannelFacebookPageFansGenderAge MemoryChannelFacebookPageFans MemoryChannelFacebookPagePosts MemoryChannelFacebookPageViews
FacebookAgent.sinks = HDFSFacebookPageFansCity HDFSFacebookPageFansGenderAge HDFSFacebookPageFans HDFSFacebookPagePosts HDFSFacebookPageViews
# FacebookPageFansCity
FacebookAgent.sources.FacebookPageFansCity.type = br.com.tsystems.hadoop.flume.source.restfb.FacebookPageFansCitySource
FacebookAgent.sources.FacebookPageFansCity.channels = MemoryChannelFacebookPageFansCity
FacebookAgent.sources.FacebookPageFansCity.appId = null
FacebookAgent.sources.FacebookPageFansCity.appSecret = null
FacebookAgent.sources.FacebookPageFansCity.accessToken = *confidential*
FacebookAgent.sources.FacebookPageFansCity.pageId = *confidential*
FacebookAgent.sources.FacebookPageFansCity.proxyEnabled = false
FacebookAgent.sources.FacebookPageFansCity.proxyHost = null
FacebookAgent.sources.FacebookPageFansCity.proxyPort = -1
FacebookAgent.sources.FacebookPageFansCity.refreshInterval = 3600
FacebookAgent.sinks.HDFSFacebookPageFansCity.channel = MemoryChannelFacebookPageFansCity
FacebookAgent.sinks.HDFSFacebookPageFansCity.type = hdfs
FacebookAgent.sinks.HDFSFacebookPageFansCity.hdfs.path = hdfs://hdoop01:8020/user/flume/pocfacebook/pagefanscity/%Y%m%d%H
FacebookAgent.sinks.HDFSFacebookPageFansCity.hdfs.fileType = DataStream
FacebookAgent.sinks.HDFSFacebookPageFansCity.hdfs.writeFormat = Text
FacebookAgent.sinks.HDFSFacebookPageFansCity.hdfs.batchSize = 1000
FacebookAgent.sinks.HDFSFacebookPageFansCity.hdfs.rollSize = 0
FacebookAgent.sinks.HDFSFacebookPageFansCity.hdfs.rollCount = 10000
FacebookAgent.channels.MemoryChannelFacebookPageFansCity.type = memory
FacebookAgent.channels.MemoryChannelFacebookPageFansCity.capacity = 10000
FacebookAgent.channels.MemoryChannelFacebookPageFansCity.transactionCapacity = 1000
# FacebookPageFansGenderAge
FacebookAgent.sources.FacebookPageFansGenderAge.type = br.com.tsystems.hadoop.flume.source.restfb.FacebookPageFansGenderAgeSource
FacebookAgent.sources.FacebookPageFansGenderAge.channels = MemoryChannelFacebookPageFansGenderAge
FacebookAgent.sources.FacebookPageFansGenderAge.appId = null
FacebookAgent.sources.FacebookPageFansGenderAge.appSecret = null
FacebookAgent.sources.FacebookPageFansGenderAge.accessToken = *confidential*
FacebookAgent.sources.FacebookPageFansGenderAge.pageId = *confidential*
FacebookAgent.sources.FacebookPageFansGenderAge.proxyEnabled = false
FacebookAgent.sources.FacebookPageFansGenderAge.proxyHost = null
FacebookAgent.sources.FacebookPageFansGenderAge.proxyPort = -1
FacebookAgent.sources.FacebookPageFansGenderAge.refreshInterval = 3600
FacebookAgent.sinks.HDFSFacebookPageFansGenderAge.channel = MemoryChannelFacebookPageFansGenderAge
FacebookAgent.sinks.HDFSFacebookPageFansGenderAge.type = hdfs
FacebookAgent.sinks.HDFSFacebookPageFansGenderAge.hdfs.path = hdfs://hdoop01:8020/user/flume/pocfacebook/pagefansgenderage/%Y%m%d%H
FacebookAgent.sinks.HDFSFacebookPageFansGenderAge.hdfs.fileType = DataStream
FacebookAgent.sinks.HDFSFacebookPageFansGenderAge.hdfs.writeFormat = Text
FacebookAgent.sinks.HDFSFacebookPageFansGenderAge.hdfs.batchSize = 1000
FacebookAgent.sinks.HDFSFacebookPageFansGenderAge.hdfs.rollSize = 0
FacebookAgent.sinks.HDFSFacebookPageFansGenderAge.hdfs.rollCount = 10000
FacebookAgent.channels.MemoryChannelFacebookPageFansGenderAge.type = memory
FacebookAgent.channels.MemoryChannelFacebookPageFansGenderAge.capacity = 10000
FacebookAgent.channels.MemoryChannelFacebookPageFansGenderAge.transactionCapacity = 1000
# FacebookPageFans
FacebookAgent.sources.FacebookPageFans.type = br.com.tsystems.hadoop.flume.source.restfb.FacebookPageFansSource
FacebookAgent.sources.FacebookPageFans.channels = MemoryChannelFacebookPageFans
FacebookAgent.sources.FacebookPageFans.appId = null
FacebookAgent.sources.FacebookPageFans.appSecret = null
FacebookAgent.sources.FacebookPageFans.accessToken = *confidential*
FacebookAgent.sources.FacebookPageFans.pageId = *confidential*
FacebookAgent.sources.FacebookPageFans.proxyEnabled = false
FacebookAgent.sources.FacebookPageFans.proxyHost = null
FacebookAgent.sources.FacebookPageFans.proxyPort = -1
FacebookAgent.sources.FacebookPageFans.refreshInterval = 3600
FacebookAgent.sinks.HDFSFacebookPageFans.channel = MemoryChannelFacebookPageFans
FacebookAgent.sinks.HDFSFacebookPageFans.type = hdfs
FacebookAgent.sinks.HDFSFacebookPageFans.hdfs.path = hdfs://hdoop01:8020/user/flume/pocfacebook/pagefans/%Y%m%d%H
FacebookAgent.sinks.HDFSFacebookPageFans.hdfs.fileType = DataStream
FacebookAgent.sinks.HDFSFacebookPageFans.hdfs.writeFormat = Text
FacebookAgent.sinks.HDFSFacebookPageFans.hdfs.batchSize = 1000
FacebookAgent.sinks.HDFSFacebookPageFans.hdfs.rollSize = 0
FacebookAgent.sinks.HDFSFacebookPageFans.hdfs.rollCount = 10000
FacebookAgent.channels.MemoryChannelFacebookPageFans.type = memory
FacebookAgent.channels.MemoryChannelFacebookPageFans.capacity = 10000
FacebookAgent.channels.MemoryChannelFacebookPageFans.transactionCapacity = 1000
# FacebookPagePosts
FacebookAgent.sources.FacebookPagePosts.type = br.com.tsystems.hadoop.flume.source.restfb.FacebookPagePostsSource
FacebookAgent.sources.FacebookPagePosts.channels = MemoryChannelFacebookPagePosts
FacebookAgent.sources.FacebookPagePosts.appId = null
FacebookAgent.sources.FacebookPagePosts.appSecret = null
FacebookAgent.sources.FacebookPagePosts.accessToken = *confidential*
FacebookAgent.sources.FacebookPagePosts.pageId = *confidential*
FacebookAgent.sources.FacebookPagePosts.proxyEnabled = false
FacebookAgent.sources.FacebookPagePosts.proxyHost = null
FacebookAgent.sources.FacebookPagePosts.proxyPort = -1
FacebookAgent.sources.FacebookPagePosts.refreshInterval = 3600
FacebookAgent.sinks.HDFSFacebookPagePosts.channel = MemoryChannelFacebookPagePosts
FacebookAgent.sinks.HDFSFacebookPagePosts.type = hdfs
FacebookAgent.sinks.HDFSFacebookPagePosts.hdfs.path = hdfs://hdoop01:8020/user/flume/pocfacebook/pageposts/%Y%m%d%H
FacebookAgent.sinks.HDFSFacebookPagePosts.hdfs.fileType = DataStream
FacebookAgent.sinks.HDFSFacebookPagePosts.hdfs.writeFormat = Text
FacebookAgent.sinks.HDFSFacebookPagePosts.hdfs.batchSize = 1000
FacebookAgent.sinks.HDFSFacebookPagePosts.hdfs.rollSize = 0
FacebookAgent.sinks.HDFSFacebookPagePosts.hdfs.rollCount = 10000
FacebookAgent.channels.MemoryChannelFacebookPagePosts.type = memory
FacebookAgent.channels.MemoryChannelFacebookPagePosts.capacity = 10000
FacebookAgent.channels.MemoryChannelFacebookPagePosts.transactionCapacity = 5000
# FacebookPageViews
FacebookAgent.sources.FacebookPageViews.type = br.com.tsystems.hadoop.flume.source.restfb.FacebookPageViewsSource
FacebookAgent.sources.FacebookPageViews.channels = MemoryChannelFacebookPageViews
FacebookAgent.sources.FacebookPageViews.appId = null
FacebookAgent.sources.FacebookPageViews.appSecret = null
FacebookAgent.sources.FacebookPageViews.accessToken = *confidential*
FacebookAgent.sources.FacebookPageViews.pageId = *confidential*
FacebookAgent.sources.FacebookPageViews.proxyEnabled = false
FacebookAgent.sources.FacebookPageViews.proxyHost = null
FacebookAgent.sources.FacebookPageViews.proxyPort = -1
FacebookAgent.sources.FacebookPageViews.refreshInterval = 3600
FacebookAgent.sinks.HDFSFacebookPageViews.channel = MemoryChannelFacebookPageViews
FacebookAgent.sinks.HDFSFacebookPageViews.type = hdfs
FacebookAgent.sinks.HDFSFacebookPageViews.hdfs.path = hdfs://hdoop01:8020/user/flume/pocfacebook/pageviews/%Y%m%d%H
FacebookAgent.sinks.HDFSFacebookPageViews.hdfs.fileType = DataStream
FacebookAgent.sinks.HDFSFacebookPageViews.hdfs.writeFormat = Text
FacebookAgent.sinks.HDFSFacebookPageViews.hdfs.batchSize = 1000
FacebookAgent.sinks.HDFSFacebookPageViews.hdfs.rollSize = 0
FacebookAgent.sinks.HDFSFacebookPageViews.hdfs.rollCount = 10000
FacebookAgent.channels.MemoryChannelFacebookPageViews.type = memory
FacebookAgent.channels.MemoryChannelFacebookPageViews.capacity = 10000
FacebookAgent.channels.MemoryChannelFacebookPageViews.transactionCapacity = 1000
Can anybody help me?
UPDATE
My coordinator from Oozie
Below is my code.. I'm trying to display each of the seq using celldisp..but it comes out one by one..
%mycode
seq1 = {'acc';'gct';'tta';'ccc';'ttc';'aaa';'ttg';'gta';'gtg';'act'};
seq2 = {'act';'gcc';'tta';'cca';'aac';'aaa';'ttg';'gta';'gtg';'acc'};
seq3 = {'ttc';'tgc';'atg';'ggc';'ccg';'aat';'aag';'ggt';'tga';'agt'};
seq4 = {'cag';'ccg';'ttg';'caa';'aat';'agc';'ttg';'ggg';'gct';'att'};
seq5 = {'ccc';'ggg';'tta';'cag';'aac';'aaa';'ttg';'gta';'gac';'acc'};
seq6 = {'acc';'gct';'ata';'ccc';'ttc';'taa';'ttg';'gtc';'gtg';'acc'};
for mutseq = {seq2,seq3,seq4,seq5,seq6}
a = strcmp(seq1,mutseq);
celldisp(mutseq);
end
%endcode
the output is :
mutseq{1}{1} =
act
mutseq{1}{2} =
gcc
mutseq{1}{3} =
tta
mutseq{1}{4} =
cca
mutseq{1}{5} =
aac
mutseq{1}{6} =
aaa
mutseq{1}{7} =
ttg
mutseq{1}{8} =
gta
mutseq{1}{9} =
gtg
mutseq{1}{10} =
%and it repeat for next seq
did anyone know how to combine it into one?