Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: missing user policy enforcement in PostPolicyHandler #11682

Merged
merged 1 commit into from Mar 3, 2021
Merged

fix: missing user policy enforcement in PostPolicyHandler #11682

merged 1 commit into from Mar 3, 2021

Conversation

harshavardhana
Copy link
Member

Description

fix: missing user policy enforcement in PostPolicyHandler

Motivation and Context

PostPolicyHandler did not honor the user policies

How to test this PR?

Manually using

mc alias set nio http://127.0.0.1:9000 nio_admin nio_admin

mc mb nio/mybucket

mc admin user add nio alice alice123
mc alias set nio_alice http://127.0.0.1:9000 alice alice123
cat > r_mybucket.json << EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::mybucket/*"
            ]
        }
    ]
}
EOF
mc admin policy add nio rMybucket r_mybucket.json
mc admin policy set nio rMybucket user=alice

mc ls nio_alice/mybucket
mc cp file.png nio_alice/mybucket/
>> mc: <ERROR> Failed to copy `file.png`. Insufficient permissions to access  this file `http://127.0.0.1:9000/mybucket/file.png`

crucial part

mc share upload  --expire 10m nio_alice/mybucket/file.png
#> URL: http://127.0.0.1:9000/mybucket/file.png
#> Expire: 10 minutes 0 seconds
#> Share: curl http://127.0.0.1:9000/mybucket/ -F x-amz-algorithm=AWS4-HMAC-SHA256 -F x-amz-credential=alice/20210216/us-east-1/s3/aws4_request -F x-amz-date=20210216T132315Z -F x-amz-signature=8ec77bb40ab98baa711aded91bf0e79827fb0335820789e95f03db6be29d57e3 -F bucket=mybucket -F policy=eyJleHBpcmF0aW9uIjoiMjAyMS0wMi0xNlQxMzozMzoxNS4wMTdaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIkYnVja2V0IiwibXlidWNrZXQiXSxbImVxIiwiJGtleSIsImZpbGUucG5nIl0sWyJlcSIsIiR4LWFtei1kYXRlIiwiMjAyMTAyMTZUMTMyMzE1WiJdLFsiZXEiLCIkeC1hbXotYWxnb3JpdGhtIiwiQVdTNC1ITUFDLVNIQTI1NiJdLFsiZXEiLCIkeC1hbXotY3JlZGVudGlhbCIsImFsaWNlLzIwMjEwMjE2L3VzLWVhc3QtMS9zMy9hd3M0X3JlcXVlc3QiXV19 -F key=file.png -F file=@<FILE>

curl http://127.0.0.1:9000/mybucket/ -F x-amz-algorithm=AWS4-HMAC-SHA256 -F x-amz-credential=alice/20210216/us-east-1/s3/aws4_request -F x-amz-date=20210216T132315Z -F x-amz-signature=8ec77bb40ab98baa711aded91bf0e79827fb0335820789e95f03db6be29d57e3 -F bucket=mybucket -F policy=eyJleHBpcmF0aW9uIjoiMjAyMS0wMi0xNlQxMzozMzoxNS4wMTdaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIkYnVja2V0IiwibXlidWNrZXQiXSxbImVxIiwiJGtleSIsImZpbGUucG5nIl0sWyJlcSIsIiR4LWFtei1kYXRlIiwiMjAyMTAyMTZUMTMyMzE1WiJdLFsiZXEiLCIkeC1hbXotYWxnb3JpdGhtIiwiQVdTNC1ITUFDLVNIQTI1NiJdLFsiZXEiLCIkeC1hbXotY3JlZGVudGlhbCIsImFsaWNlLzIwMjEwMjE2L3VzLWVhc3QtMS9zMy9hd3M0X3JlcXVlc3QiXV19 -F key=file.png -F file=@file.png

mc ls nio_alice/mybucket
[2021-02-16 14:23:42 CET] 135KiB file.png

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Documentation updated
  • Unit tests added/updated

vadmeste
vadmeste reviewed Mar 2, 2021
View reviewed changes
Copy link
Member

@vadmeste vadmeste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments

cmd/bucket-handlers.go Show resolved Hide resolved
cmd/bucket-handlers.go Show resolved Hide resolved
Alevsk
Alevsk approved these changes Mar 2, 2021
View reviewed changes
Copy link
Contributor

@Alevsk Alevsk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested, works fine

@minio-trusted
Copy link
Contributor

Mint Automation

Test Result
mint-large-bucket.sh ✔️
mint-fs.sh ✔️
mint-gateway-s3.sh ✔️
mint-erasure.sh ✔️
mint-dist-erasure.sh ✔️
mint-zoned.sh ✔️
mint-gateway-nas.sh ❌ more...
mint-compress-encrypt-dist-erasure.sh ❌ more...

11682-5d4daad/mint-compress-encrypt-dist-erasure.sh.log:

Running with
SERVER_ENDPOINT:      minio-c3.minio.io:32226
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp f8e9ed17ef08:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 2 seconds
(2/15) Running aws-sdk-java tests ... done in 1 seconds
(3/15) Running aws-sdk-php tests ... done in 43 seconds
(4/15) Running aws-sdk-ruby tests ... done in 4 seconds
(5/15) Running awscli tests ... FAILED in 32 seconds
{
  "name": "awscli",
  "duration": 2594,
  "function": "aws --endpoint-url http://minio-c3.minio.io:32226 s3api copy-object --bucket awscli-mint-test-bucket-17628 --key datafile-1-kB-copy --copy-source awscli-mint-test-bucket-17628/datafile-1-kB\n",
  "status": "FAIL",
  "error": "Hash mismatch expected 084e1383b70fb0c51acc680fef370023, got ac57de7156d7fc25ac1a65f81fa3989b"
}
(5/15) Running healthcheck tests ... done in 1 seconds
(6/15) Running mc tests ... done in 50 seconds
(7/15) Running minio-dotnet tests ... done in 48 seconds
(8/15) Running minio-go tests ... FAILED in 2 minutes and 42 seconds
{
  "args": {
    "destination": {
      "Bucket": "minio-go-test-tlbgqrfnwx42d20n",
      "Object": "dstObject",
      "Encryption": {},
      "UserMetadata": null,
      "ReplaceMetadata": false,
      "UserTags": null,
      "ReplaceTags": false,
      "LegalHold": "",
      "Mode": "",
      "RetainUntilDate": "0001-01-01T00:00:00Z",
      "Size": 0,
      "Progress": null
    },
    "source": {
      "Bucket": "minio-go-test-tlbgqrfnwx42d20n",
      "Object": "srcObject",
      "VersionID": "",
      "MatchETag": "",
      "NoMatchETag": "",
      "MatchModifiedSince": "0001-01-01T00:00:00Z",
      "MatchUnmodifiedSince": "0001-01-01T00:00:00Z",
      "MatchRange": false,
      "Start": 0,
      "End": 0,
      "Encryption": null
    }
  },
  "duration": 7896,
  "error": "We encountered an internal error, please try again.: cause(s2: corrupt input)",
  "function": "CopyObject(destination, source)",
  "message": "GetObject failed",
  "name": "minio-go: testUnencryptedToSSES3CopyObject",
  "status": "FAIL"
}
(8/15) Running minio-java tests ... FAILED in 1 minutes and 37 seconds
{
  "name": "minio-java",
  "function": "copyObject()",
  "args": "[match etag]",
  "duration": 329,
  "status": "FAIL",
  "error": "error occurred\nErrorResponse(code = PreconditionFailed, message = At least one of the pre-conditions you specified did not hold, bucketName = minio-java-test-2qelgi6, objectName = minio-java-test-189pp9-copy, resource = /minio-java-test-2qelgi6/minio-java-test-189pp9-copy, requestId = 1668ADC9E824BCD8, hostId = 22b9e7dd-e361-4da7-9000-a08ee0af4e95)\nrequest={method=PUT, url=http://minio-c3.minio.io:32226/minio-java-test-2qelgi6/minio-java-test-189pp9-copy, headers=x-amz-copy-source-if-match: 71cff0a060f852067e443ad1e24ae26c-1\nx-amz-copy-source: /minio-java-test-4oqb3i/minio-java-test-189pp9\nHost: minio-c3.minio.io:32226\nAccept-Encoding: identity\nUser-Agent: MinIO (Linux; amd64) minio-java/8.0.3\nContent-MD5: 1B2M2Y8AsgTpgAmY7PhCfg==\nx-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date: 20210303T003229Z\nAuthorization: AWS4-HMAC-SHA256 Credential=*REDACTED*/20210303/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-copy-source;x-amz-copy-source-if-match;x-amz-date, Signature=*REDACTED*\n}\nresponse={code=412, headers=Accept-Ranges: bytes\nContent-Length: 416\nContent-Security-Policy: block-all-mixed-content\nContent-Type: application/xml\nETag: \"71cff0a060f852067e443ad1e24ae26c\"\nLast-Modified: Wed, 03 Mar 2021 00:32:28 GMT\nServer: MinIO\nVary: Origin\nX-Amz-Request-Id: 1668ADC9E824BCD8\nX-Xss-Protection: 1; mode=block\nDate: Wed, 03 Mar 2021 00:32:29 GMT\n}\n >>> [io.minio.MinioClient.execute(MinioClient.java:775), io.minio.MinioClient.execute(MinioClient.java:563), io.minio.MinioClient.executePut(MinioClient.java:904), io.minio.MinioClient.copyObject(MinioClient.java:1232), FunctionalTest.testCopyObjectMatchETag(FunctionalTest.java:1850), FunctionalTest.copyObject(FunctionalTest.java:2016), FunctionalTest.runObjectTests(FunctionalTest.java:3757), FunctionalTest.runTests(FunctionalTest.java:3783), FunctionalTest.main(FunctionalTest.java:3927)]"
}
(8/15) Running minio-js tests ... done in 53 seconds
(9/15) Running minio-py tests ... done in 3 minutes and 13 seconds
(10/15) Running s3cmd tests ... FAILED in 6 seconds
{
  "name": "s3cmd",
  "duration": "3111",
  "function": "test_put_object_multipart",
  "status": "FAIL",
  "error": "WARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Retrying upload of /mint/data/datafile-65-MB\nWARNING: MD5 Sums don't match!\nWARNING: Too many failures. Giving up on '/mint/data/datafile-65-MB'\nERROR: \nUpload of '/mint/data/datafile-65-MB' part 1 failed. Use\n  /usr/local/bin/s3cmd abortmp s3://s3cmd-test-bucket-26109/s3cmd-test-object-13780 497ce7fc-f1fa-43d2-aea3-0d0a1294cda7\nto abort the upload, or\n  /usr/local/bin/s3cmd --upload-id 497ce7fc-f1fa-43d2-aea3-0d0a1294cda7 put ...\nto continue the upload.\nERROR: Upload of '/mint/data/datafile-65-MB' failed too many times (Last reason: )"
}
(10/15) Running s3select tests ... done in 7 seconds
(11/15) Running security tests ... done in 0 seconds

Executed 11 out of 15 tests successfully.

11682-5d4daad/mint-gateway-nas.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev6.minio.io:31689
ACCESS_KEY:           minio
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 5afe0018f2b7:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 0 seconds
(2/15) Running aws-sdk-java tests ... done in 2 seconds
(3/15) Running aws-sdk-php tests ... done in 41 seconds
(4/15) Running aws-sdk-ruby tests ... done in 2 seconds
(5/15) Running awscli tests ... done in 1 minutes and 9 seconds
(6/15) Running healthcheck tests ... done in 1 seconds
(7/15) Running mc tests ... done in 44 seconds
(8/15) Running minio-dotnet tests ... done in 35 seconds
(9/15) Running minio-go tests ... done in 1 minutes and 36 seconds
(10/15) Running minio-java tests ... done in 12 seconds
(11/15) Running minio-js tests ... done in 43 seconds
(12/15) Running minio-py tests ... FAILED in 1 minutes and 0 seconds
{
  "name": "minio-py:test_presigned_get_object_expiry",
  "status": "FAIL",
  "args": {
    "bucket_name": "minio-py-test-f43ce1ac-2fb3-490e-96e1-e8a9f2e93ae1",
    "object_name": "3971a2ca-8a3b-4840-9be1-9d91718325fa",
    "presigned_get_object_url": "http://minio-dev6.minio.io:31689/minio-py-test-f43ce1ac-2fb3-490e-96e1-e8a9f2e93ae1/3971a2ca-8a3b-4840-9be1-9d91718325fa?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20210303%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210303T003154Z&X-Amz-Expires=120&X-Amz-SignedHeaders=host&X-Amz-Signature=3aac22c536484611b55ef21b295fd3cfbc6bc417272a1c5d9382203d4482487a",
    "response.status": 200,
    "response.reason": "OK",
    "response.headers": "{\"_container\": {\"accept-ranges\": [\"Accept-Ranges\", \"bytes\"], \"content-length\": [\"Content-Length\", \"1024\"], \"content-security-policy\": [\"Content-Security-Policy\", \"block-all-mixed-content\"], \"content-type\": [\"Content-Type\", \"application/octet-stream\"], \"etag\": [\"ETag\", \"\\\"1eeaad3aba4f7d2a5fa4646187d08d6e\\\"\"], \"last-modified\": [\"Last-Modified\", \"Wed, 03 Mar 2021 00:31:52 GMT\"], \"server\": [\"Server\", \"MinIO\"], \"vary\": [\"Vary\", \"Origin\"], \"x-amz-request-id\": [\"X-Amz-Request-Id\", \"1668ADC17B68AFA2\"], \"x-xss-protection\": [\"X-Xss-Protection\", \"1; mode=block\"], \"date\": [\"Date\", \"Wed, 03 Mar 2021 00:31:52 GMT\"]}}",
    "response._body": "rfbxycswemzzswlutqeobtfmvqjsxpqgwehgoyhlafplhwojoibmetoriqpcctkuahblmlpmrsjoomjpbkzzaywlpeciwusbqzoxvashaczndbjcnawkyiqhmlmkkqkhiwywbidcjjenfmagutsagmygcucfguzzxjoodncisexayyblqkhkegatibxqhlpcunqyrmoheiyuuprkmwnrjdsivtrlywqypdfveeykgbicrvhyxbtxpwospievhqocxwfnkqnvzewywuxmnvxvrmfqystggfkvvacdsosyvcbdtrlgubsqnkmonxbvqzgcfdhmcbivccsrbqowaahnfjlphzgxgmhwsrruuprvukqgovwdnxugpuzywbkqfejmuoexdhgkdxrvkdlybqbmehkfeyfgaomikqnisoyvqlpfernhivqwkppwpzkhblrliazijvaddlvpnemleraxkclyjzledudjqhejgqdessrqiyfuhplffkxzjgwfmlgzsnizvigkfwkgjsvonottvvqhnesroaztaciuezuyxxkzljxfsjqptzbmruxpnefcrldzzsdbjgjjamgiwzcgfndjezwteoxmdhkoimpbruhtddewvcfhujtfuslncresgxfpninttdkhotjgjvsbeuuprevtdnhtgnblewrijhhfssifinfooufkdnjdscyauktgaptztmowhlcmdhtiyxvzbhjaqcecmpzaywblrmyhidpujwvigclcsnjaxdtaqvdterkupocebdjiqtceiasfltfdozwynjlqpwyakluknbyurubaazdwbsnoxrttazcaxqketcrdjgaeccpamkkdauqpplzyxrxffhlrjskjxtnefjisoqpuufmgyriztvztxhanwbyihofmvlkqgbfwqydwytdgcjjdcmizprxhjqcxpfbzusyyasxblsrrnirgzxwltxfpvwivaaxwnonnigbqhoypmmlhgogaxorvvptukvbmmobdvgblnzms",
    "response.status-2": 200,
    "response.reason-2": "OK",
    "response.headers-2": "{\"_container\": {\"accept-ranges\": [\"Accept-Ranges\", \"bytes\"], \"content-length\": [\"Content-Length\", \"1024\"], \"content-security-policy\": [\"Content-Security-Policy\", \"block-all-mixed-content\"], \"content-type\": [\"Content-Type\", \"application/octet-stream\"], \"etag\": [\"ETag\", \"\\\"1eeaad3aba4f7d2a5fa4646187d08d6e\\\"\"], \"last-modified\": [\"Last-Modified\", \"Wed, 03 Mar 2021 00:31:52 GMT\"], \"server\": [\"Server\", \"MinIO\"], \"vary\": [\"Vary\", \"Origin\"], \"x-amz-request-id\": [\"X-Amz-Request-Id\", \"1668ADC1F325D8F1\"], \"x-xss-protection\": [\"X-Xss-Protection\", \"1; mode=block\"], \"date\": [\"Date\", \"Wed, 03 Mar 2021 00:31:54 GMT\"]}}",
    "response._body-2": "rfbxycswemzzswlutqeobtfmvqjsxpqgwehgoyhlafplhwojoibmetoriqpcctkuahblmlpmrsjoomjpbkzzaywlpeciwusbqzoxvashaczndbjcnawkyiqhmlmkkqkhiwywbidcjjenfmagutsagmygcucfguzzxjoodncisexayyblqkhkegatibxqhlpcunqyrmoheiyuuprkmwnrjdsivtrlywqypdfveeykgbicrvhyxbtxpwospievhqocxwfnkqnvzewywuxmnvxvrmfqystggfkvvacdsosyvcbdtrlgubsqnkmonxbvqzgcfdhmcbivccsrbqowaahnfjlphzgxgmhwsrruuprvukqgovwdnxugpuzywbkqfejmuoexdhgkdxrvkdlybqbmehkfeyfgaomikqnisoyvqlpfernhivqwkppwpzkhblrliazijvaddlvpnemleraxkclyjzledudjqhejgqdessrqiyfuhplffkxzjgwfmlgzsnizvigkfwkgjsvonottvvqhnesroaztaciuezuyxxkzljxfsjqptzbmruxpnefcrldzzsdbjgjjamgiwzcgfndjezwteoxmdhkoimpbruhtddewvcfhujtfuslncresgxfpninttdkhotjgjvsbeuuprevtdnhtgnblewrijhhfssifinfooufkdnjdscyauktgaptztmowhlcmdhtiyxvzbhjaqcecmpzaywblrmyhidpujwvigclcsnjaxdtaqvdterkupocebdjiqtceiasfltfdozwynjlqpwyakluknbyurubaazdwbsnoxrttazcaxqketcrdjgaeccpamkkdauqpplzyxrxffhlrjskjxtnefjisoqpuufmgyriztvztxhanwbyihofmvlkqgbfwqydwytdgcjjdcmizprxhjqcxpfbzusyyasxblsrrnirgzxwltxfpvwivaaxwnonnigbqhoypmmlhgogaxorvvptukvbmmobdvgblnzms"
  },
  "message": "Presigned get url failed to expire!",
  "error": "Traceback (most recent call last):\n  File \"/mint/run/core/minio-py/tests.py\", line 126, in _call_test\n    func(log_entry, *args, **kwargs)\n  File \"/mint/run/core/minio-py/tests.py\", line 1364, in test_presigned_get_object_expiry\n    raise ValueError('Presigned get url failed to expire!')\nValueError: Presigned get url failed to expire!\n",
  "duration": 2039
}
(12/15) Running s3cmd tests ... done in 16 seconds
(13/15) Running s3select tests ... done in 4 seconds
(14/15) Running security tests ... done in 0 seconds

Executed 14 out of 15 tests successfully.

Deleting image on docker hub
Deleting image locally

@harshavardhana harshavardhana merged commit 039f59b into minio:master Mar 3, 2021
@harshavardhana harshavardhana deleted the policy-enforcement branch March 3, 2021 16:47
harshavardhana added a commit that referenced this pull request May 18, 2021
@harshavardhana
fix: missing user policy enforcement in PostPolicyHandler (#11682)
d04201e
@tatianab tatianab mentioned this pull request Nov 8, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Alevsk Alevsk Alevsk approved these changes

@aead aead aead approved these changes

@vadmeste vadmeste Awaiting requested review from vadmeste

Assignees
No one assigned
Labels
priority: high
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@harshavardhana @minio-trusted @vadmeste @Alevsk @aead