radosgw swift interface put object response 500 error? - ceph

Question

I startup a radosgw (Civetweb embedded), i want use swift interface to put/get oject;
my radosgw's ceph config：
[global]
fsid = 584ba99d-0665-4465-b693-6c78ae25576f
mon_initial_members = n6-0**-0**, n6-0**-0**, n6-0**-0**
mon_host = 10.6.**.**, 10.5.**.**, 10.4.**.**
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true
osd pool default size = 1 # Write an object 3 times.
osd pool default min size = 1 # Allow writing one copy in a degraded state.
osd pool default pg num = 2000
osd pool default pgp num = 2000
keyring = /etc/ceph/ceph.client.admin.keyring
#debug ms = 1
#debug rgw = 20
[client.radosgw.gateway]
host = in**-**
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw_frontends = "civetweb port=9980"
rgw socket path = ""
log file = /var/log/radosgw/client.bootstrap-rgw.log
radosgw.keyring
ceph# vi ceph.client.radosgw.keyring
1 [client.radosgw.gateway]
2 key = AQDY5pFXyvELDRAAvl6HCERMwpwfHIKaA23rlw==
start radosgw:
/etc/init.d/radosgw start
secondly, i create a swift user:
radosgw-admin user create --uid="dockeruser" --display-name="Docker User"
radosgw-admin subuser create --uid=dockeruser --subuser=dockeruser:swift --access=full
radosgw-admin key create --subuser=dockeruser:swift --key-type=swift --gen-secret
{
"user_id": "dockeruser",
"display_name": "Docker User",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"auid": 0,
"subusers": [
{
"id": "dockeruser:swift",
"permissions": "full-control"
}
],
"keys": [
{
"user": "dockeruser",
"access_key": "9LJMM0BAMI7XU1ZAB0BG",
"secret_key": "8uF7OLlLsqCaCsfE08sOKiCb9gIrYEQICoH475Xw"
}
],
"swift_keys": [
{
"user": "dockeruser:swift",
"secret_key": "oY894WlkjlyUAxHacYNMAyR8dpR3ZzlRoBJbt3xW"
}
],
"caps": [],
"op_mask": "read, write, delete",
"default_placement": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
},
"temp_url_keys": []
}
and then use swiftclient to communicate:
swiftclient$ python shell.py -A http://10.4.**.**:9980/auth/1.0 -U dockeruser:swift -K 'oY894WlkjlyUAxHacYNMAyR8dpR3ZzlRoBJbt3xW' stat
Account: v1
Containers: 5
Objects: 0
Bytes: 0
X-Timestamp: 1469590199.89180
X-Account-Bytes-Used-Actual: 0
X-Trans-Id: tx0000000000000000012e8-0057982ab7-b215dd-default
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
create a bucket(container):
swiftclient$ python shell.py -A http://10.4.**.**:9980/auth/1.0 -U dockeruser:swift -K 'oY894WlkjlyUAxHacYNMAyR8dpR3ZzlRoBJbt3xW' post test1
list container( everything works well):
swiftclient$ python shell.py -A http://10.4.**.**:9980/auth/1.0 -U dockeruser:swift -K 'oY894WlkjlyUAxHacYNMAyR8dpR3ZzlRoBJbt3xW' list
colin
my-new-bucket
registry
registry22
test1
but when i put object(use python scripts), i got error:
swiftclient$ ./put_object.py
Traceback (most recent call last):
File "./put_object.py", line 33, in <module>
with urllib.request.urlopen(req, binary_data) as f:
File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.4/urllib/request.py", line 461, in open
response = meth(req, response)
File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.4/urllib/request.py", line 499, in error
return self._call_chain(*args)
File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
result = func(*args)
File "/usr/lib/python3.4/urllib/request.py", line 579, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: Internal Server Error
radosgw's log：
2016-07-27 14:06:32.032983 7f5f0cff9700 2 req 1:0.056814:swift:PUT /swift/v1/registry22/colin_key:put_obj:recalculating target
2016-07-27 14:06:32.032986 7f5f0cff9700 2 req 1:0.056817:swift:PUT /swift/v1/registry22/colin_key:put_obj:reading permissions
2016-07-27 14:06:32.032989 7f5f0cff9700 2 req 1:0.056820:swift:PUT /swift/v1/registry22/colin_key:put_obj:init op
2016-07-27 14:06:32.032996 7f5f0cff9700 2 req 1:0.056827:swift:PUT /swift/v1/registry22/colin_key:put_obj:verifying op mask
2016-07-27 14:06:32.032998 7f5f0cff9700 20 required_mask= 2 user.op_mask=7
2016-07-27 14:06:32.032999 7f5f0cff9700 2 req 1:0.056830:swift:PUT /swift/v1/registry22/colin_key:put_obj:verifying op permissions
2016-07-27 14:06:32.033002 7f5f0cff9700 5 Searching permissions for uid=dockeruser mask=50
2016-07-27 14:06:32.033003 7f5f0cff9700 5 Found permission: 15
2016-07-27 14:06:32.033004 7f5f0cff9700 5 Searching permissions for group=1 mask=50
2016-07-27 14:06:32.033005 7f5f0cff9700 5 Permissions for group not found
2016-07-27 14:06:32.033006 7f5f0cff9700 5 Searching permissions for group=2 mask=50
2016-07-27 14:06:32.033007 7f5f0cff9700 5 Permissions for group not found
2016-07-27 14:06:32.033007 7f5f0cff9700 5 Getting permissions id=dockeruser owner=dockeruser perm=2
2016-07-27 14:06:32.033008 7f5f0cff9700 10 uid=dockeruser requested perm (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
2016-07-27 14:06:32.033010 7f5f0cff9700 2 req 1:0.056841:swift:PUT /swift/v1/registry22/colin_key:put_obj:verifying op params
2016-07-27 14:06:32.033011 7f5f0cff9700 2 req 1:0.056842:swift:PUT /swift/v1/registry22/colin_key:put_obj:pre-executing
2016-07-27 14:06:32.033013 7f5f0cff9700 2 req 1:0.056844:swift:PUT /swift/v1/registry22/colin_key:put_obj:executing
2016-07-27 14:06:32.033131 7f5f0cff9700 20 get_obj_state: rctx=0x7f5f0cff2ff0 obj=registry22:colin_key state=0x7f5eac01df08 s->prefetch_data=0
2016-07-27 14:06:32.033199 7f5f0cff9700 1 -- 10.4.24.158:0/3995324651 --> 10.6.16.213:6808/2093 -- osd_op(client.7883846.0:666 19.6090fa40 a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1_colin_key [getxattrs,stat] snapc 0=[] ack+read+known_if_redirected e2967) v7 -- ?+0 0x7f5eac021e20 con 0x7f5eac020a20
2016-07-27 14:06:32.041907 7f5f0c7f8700 1 -- 10.4.24.158:0/3995324651 <== osd.57 10.6.16.213:6808/2093 1 ==== osd_op_reply(666 a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1_colin_key [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 266+0+0 (1562684643 0 0) 0x7f5cb4001ac0 con 0x7f5eac020a20
2016-07-27 14:06:32.041994 7f5f0cff9700 20 get_obj_state: rctx=0x7f5f0cff2ff0 obj=registry22:colin_key state=0x7f5eac01df08 s->prefetch_data=0
2016-07-27 14:06:32.042016 7f5f0cff9700 10 setting object write_tag=a9da190c-5df5-4978-8fc6-d411b1d5ddb3.7883846.1
2016-07-27 14:06:32.042082 7f5f0cff9700 20 reading from default.rgw.data.root:.bucket.meta.registry22:a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1
2016-07-27 14:06:32.042091 7f5f0cff9700 20 get_system_obj_state: rctx=0x7f5f0cff1c00 obj=default.rgw.data.root:.bucket.meta.registry22:a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 state=0x7f5eac02eb48 s->prefetch_data=0
2016-07-27 14:06:32.042098 7f5f0cff9700 10 cache get: name=default.rgw.data.root+.bucket.meta.registry22:a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 : type miss (requested=22, cached=19)
2016-07-27 14:06:32.042123 7f5f0cff9700 1 -- 10.4.24.158:0/3995324651 --> 10.6.16.212:6810/96264 -- osd_op(client.7883846.0:667 11.3988fae5 .bucket.meta.registry22:a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 [call version.read,getxattrs,stat] snapc 0=[] ack+read+known_if_redirected e2967) v7 -- ?+0 0x7f5eac030af0 con 0x7f5f5800b750
2016-07-27 14:06:32.047598 7f5f644f5700 1 -- 10.4.24.158:0/3995324651 <== osd.43 10.6.16.212:6810/96264 48 ==== osd_op_reply(667 .bucket.meta.registry22:a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 [call,getxattrs,stat] v0'0 uv2 ondisk = 0) v6 ==== 322+0+318 (1672081398 0 832274712) 0x7f5cc4005be0 con 0x7f5f5800b750
2016-07-27 14:06:32.047631 7f5f0cff9700 10 cache put: name=default.rgw.data.root+.bucket.meta.registry22:a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 info.flags=22
2016-07-27 14:06:32.047639 7f5f0cff9700 10 moving default.rgw.data.root+.bucket.meta.registry22:a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 to cache LRU end
2016-07-27 14:06:32.047641 7f5f0cff9700 10 updating xattr: name=user.rgw.acl bl.length()=159
2016-07-27 14:06:32.047644 7f5f0cff9700 20 get_system_obj_state: s->obj_tag was set empty
2016-07-27 14:06:32.047647 7f5f0cff9700 10 cache get: name=default.rgw.data.root+.bucket.meta.registry22:a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 : hit (requested=17, cached=23)
2016-07-27 14:06:32.047659 7f5f0cff9700 20 bucket index object: .dir.a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1
2016-07-27 14:06:32.047727 7f5f0cff9700 1 -- 10.4.24.158:0/3995324651 --> 10.6.16.213:6812/2271 -- osd_op(client.7883846.0:668 18.29a991bc .dir.a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 [call rgw.bucket_prepare_op] snapc 0=[] ondisk+write+known_if_redirected e2967) v7 -- ?+0 0x7f5eac034520 con 0x7f5eac02edb0
2016-07-27 14:06:32.118719 7f5f0c5f6700 1 -- 10.4.24.158:0/3995324651 <== osd.53 10.6.16.213:6812/2271 1 ==== osd_op_reply(668 .dir.a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 [call] v2967'20 uv20 ondisk = 0) v6 ==== 219+0+0 (1279524478 0 0) 0x7f5cbc001ad0 con 0x7f5eac02edb0
2016-07-27 14:06:32.118789 7f5f0cff9700 1 -- 10.4.24.158:0/3995324651 --> 10.6.16.213:6808/2093 -- osd_op(client.7883846.0:669 19.6090fa40 a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1_colin_key [create 0~0 [excl],setxattr user.rgw.idtag (47),writefull 0~27,setxattr user.rgw.manifest (593),setxattr user.rgw.acl (159),setxattr user.rgw.content_type (34),setxattr user.rgw.etag (33),call rgw.obj_store_pg_ver,setxattr user.rgw.source_zone (4)] snapc 0=[] ondisk+write+known_if_redirected e2967) v7 -- ?+0 0x7f5eac031520 con 0x7f5eac020a20
2016-07-27 14:06:32.120500 7f5f0c7f8700 1 -- 10.4.24.158:0/3995324651 <== osd.57 10.6.16.213:6808/2093 2 ==== osd_op_reply(669 a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1_colin_key [create 0~0 [excl],setxattr (47),writefull 0~27,setxattr (593),setxattr (159),setxattr (34),setxattr (33),call,setxattr (4)] v0'0 uv0 ondisk = -95 ((95) Operation not supported)) v6 ==== 560+0+0 (2893445330 0 0) 0x7f5cb4000cb0 con 0x7f5eac020a20
2016-07-27 14:06:32.120586 7f5f0cff9700 1 -- 10.4.24.158:0/3995324651 --> 10.6.16.213:6812/2271 -- osd_op(client.7883846.0:670 18.29a991bc .dir.a9da190c-5df5-4978-8fc6-d411b1d5ddb3.11666571.1 [call rgw.bucket_complete_op] snapc 0=[] ack+ondisk+write+known_if_redirected e2967) v7 -- ?+0 0x7f5eac0314d0 con 0x7f5eac02edb0
2016-07-27 14:06:32.120617 7f5f0cff9700 2 req 1:0.144447:swift:PUT /swift/v1/registry22/colin_key:put_obj:completing
2016-07-27 14:06:32.120633 7f5f0cff9700 0 WARNING: set_req_state_err err_no=95 resorting to 500
2016-07-27 14:06:32.120683 7f5f0cff9700 2 req 1:0.144514:swift:PUT /swift/v1/registry22/colin_key:put_obj:op status=-95
2016-07-27 14:06:32.120687 7f5f0cff9700 2 req 1:0.144518:swift:PUT /swift/v1/registry22/colin_key:put_obj:http status=500
2016-07-27 14:06:32.120694 7f5f0cff9700 1 ====== req done req=0x7f5f0cff38a0 op status=-95 http_status=500 ======
2016-07-27 14:06:32.120707 7f5f0cff9700 20 process_request() returned -95
2016-07-27 14:06:32.120728 7f5f0cff9700 1 civetweb: 0x7f5eac0008c0: 10.6.26.137 - - [27/Jul/2016:14:06:31 +0800] "PUT /swift/v1/registry22/colin_key HTTP/1.1" 500 0 - Python-urllib/3.4
put_object's scrips:
req = Request('http://10.4.**.**:9980/swift/v1/registry22/colin_key',
method = 'PUT')
timestr = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
req.add_header('Host', '10.4.**.**')
req.add_header('Date', timestr)
#req.add_header('x-amz-acl', 'public-read-write')
req.add_header('X-Auth-Token', key)
values = {'user':'colin', 'data': 'colin_value'}
data = urllib.parse.urlencode(values)
binary_data = data.encode('utf-8')
with urllib.request.urlopen(req, binary_data) as f:
print(f.status)
print(f.read().decode('utf-8'))
i use the python scripts to list container, it works well:
swiftclient$ ./list_containers.py
200
colin
my-new-bucket
registry
registry22
test1
list_containers.py:
key = 'AUTH_rgwtk10000000646f636b6572757365723a73776966745eeb32c1f8ad71e4018f9857
4829d919ddbe49b3c2e5d3bc30ea38d48633cd8a492ba0ca'
req = Request('http://10.4.**.**:9980/swift/v1',
method = 'GET')
timestr = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
req.add_header('Host', '10.4.**.**')
req.add_header('Date', timestr)
#req.add_header('x-amz-acl', 'public-read-write')
req.add_header('X-Auth-Token', key)
with urllib.request.urlopen(req) as f:
print(f.status)
print(f.read().decode('utf-8'))
I don't know how to solve it, please help me;
thanks a lot;