{"id":1880,"date":"2012-06-13T17:49:00","date_gmt":"2012-06-13T09:49:00","guid":{"rendered":""},"modified":"2014-09-09T15:44:30","modified_gmt":"2014-09-09T07:44:30","slug":"utf-8%e7%bc%96%e7%a0%81%e4%b8%adbom%e7%9a%84%e6%a3%80%e6%b5%8b%e4%b8%8e%e5%88%a0%e9%99%a4","status":"publish","type":"post","link":"https:\/\/kyle.ai\/blog\/1880.html","title":{"rendered":"UTF-8\u7f16\u7801\u4e2dBOM\u7684\u68c0\u6d4b\u4e0e\u5220\u9664"},"content":{"rendered":"<p>\u6240\u8c13BOM\uff0c\u5168\u79f0\u662fByte Order Mark\uff0c\u5b83\u662f\u4e00\u4e2aUnicode\u5b57\u7b26\uff0c\u901a\u5e38\u51fa\u73b0\u5728\u6587\u672c\u7684\u5f00\u5934\uff0c\u7528\u6765\u6807\u8bc6\u5b57\u8282\u5e8f\uff08Big\/Little Endian\uff09\uff0c\u9664\u6b64\u4ee5\u5916\u8fd8\u53ef\u4ee5\u6807\u8bc6\u7f16\u7801\uff08UTF-8\/16\/32\uff09\uff0c\u5982\u679c\u51fa\u73b0\u5728\u6587\u672c\u4e2d\u95f4\uff0c\u5219\u89e3\u91ca\u4e3azero width no-break space\u3002 \u6ce8\uff1aUnicode\u76f8\u5173\u77e5\u8bc6\u7684\u8be6\u7ec6\u4ecb\u7ecd\u8bf7\u53c2\u8003UTF-8, UTF-16, UTF-32 &amp; BOM\u3002 \u5bf9\u4e8eUTF-8\/16\/32\u800c\u8a00\uff0c\u5b83\u4eec\u540d\u5b57\u4e2d\u76848\/16\/32\u6307\u7684\u662f\u7f16\u7801\u5355\u4f4d\u662f\u591a\u5c11\u4f4d\u7684\uff0c\u4e5f\u5c31\u662f\u8bf4\uff0c\u5b83\u4eec\u7684\u7f16\u7801\u5355\u4f4d\u5206\u522b\u662f8\/16\/32\u4f4d\uff0c\u6362\u7b97\u6210\u5b57\u8282\u5c31\u662f1\/2\/4\u5b57\u8282\uff0c\u5982\u679c\u662f\u591a\u5b57\u8282\uff0c\u5c31\u8981\u7275\u626f\u5230\u5b57\u8282\u5e8f\uff0cUTF-8\u4ee5\u5355\u5b57\u8282\u4e3a\u7f16\u7801\u5355\u4f4d\uff0c\u6240\u4ee5\u4e0d\u5b58\u5728\u5b57\u8282\u5e8f\u3002 UTF-8\u4e3b\u8981\u7684\u4f18\u70b9\u662f\u53ef\u4ee5\u517c\u5bb9ASCII\uff0c\u4f46\u5982\u679c\u4f7f\u7528BOM\u7684\u8bdd\uff0c\u8fd9\u4e2a\u597d\u5904\u5c31\u8361\u7136\u65e0\u5b58\u4e86\uff0c\u9664\u6b64\u4ee5\u5916\uff0cBOM\u7684\u5b58\u5728\u8fd8\u53ef\u80fd\u5f15\u53d1\u4e00\u4e9b\u95ee\u9898\uff0c\u6bd4\u5982\u4e0b\u9762\u9519\u8bef\u4fbf\u90fd\u6709\u53ef\u80fd\u662fBOM\u5bfc\u81f4\u7684\uff1a<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nShell: No such file or directory\r\nPHP: Warning: Cannot modify header information \u2013 headers already sent\r\n<\/pre>\n<p>\u5728\u8be6\u7ec6\u8ba8\u8bbaUTF-8\u7f16\u7801\u4e2dBOM\u7684\u68c0\u6d4b\u4e0e\u5220\u9664\u95ee\u9898\u524d\uff0c\u4e0d\u59a8\u5148\u901a\u8fc7\u4e00\u4e2a\u4f8b\u5b50\u70ed\u70ed\u8eab\uff1a<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nshell&gt; curl -s http:\/\/phone.10086.cn\/ | head -1 | sed -n l\\357\\273\\277&lt;!DOCTYPE html PUBLIC &quot;-\/\/W3C\/\/DTD XHTML 1.0 Transitional\\\/\/EN&quot; &quot;http:\/\/www.w3.org\/TR\/xhtml1\/DTD\/xhtml1-transitional.dtd&quot;&gt;\\r$\r\n<\/pre>\n<p>\u5982\u4e0a\u6240\u793a\uff0c\u524d\u4e09\u4e2a\u5b57\u8282\u5206\u522b\u662f357\u3001273\u3001277\uff0c\u8fd9\u5c31\u662f\u516b\u8fdb\u5236\u7684BOM\u3002<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nshell&gt; curl -s http:\/\/phone.10086.cn\/ | head -1 | hexdump -C00000000  ef bb bf 3c 21 44 4f 43  54 59 50 45 20 68 74 6d  |\u2026&lt;!DOCTYPE htm|00000010  6c 20 50 55 42 4c 49 43  20 22 2d 2f 2f 57 33 43  |l PUBLIC &quot;-\/\/W3C|00000020  2f 2f 44 54 44 20 58 48  54 4d 4c 20 31 2e 30 20  |\/\/DTD XHTML 1.0 |00000030  54 72 61 6e 73 69 74 69  6f 6e 61 6c 2f 2f 45 4e  |Transitional\/\/EN|00000040  22 20 22 68 74 74 70 3a  2f 2f 77 77 77 2e 77 33  |&quot; &quot;htto:\/\/www.w3|00000050  2e 6f 72 67 2f 54 52 2f  78 68 74 6d 6c 31 2f 44  |.org\/TR\/xhtml1\/D|00000060  54 44 2f 78 68 74 6d 6c  31 2d 74 72 61 6e 73 69  |TD\/xhtml1-transi|00000070  74 69 6f 6e 61 6c 2e 64  74 64 22 3e 0d 0a        |tional.dtd&quot;&gt;..|\r\n<\/pre>\n<p>\u5982\u4e0a\u6240\u793a\uff0c\u524d\u4e09\u4e2a\u5b57\u8282\u5206\u522b\u662fEF\u3001BB\u3001BF\uff0c\u8fd9\u5c31\u662f\u5341\u516d\u8fdb\u5236\u7684BOM\u3002 \u6ce8\uff1a\u7528\u5230\u4e86\u7b2c\u4e09\u65b9\u7f51\u7ad9\u7684\u9875\u9762\uff0c\u4e0d\u80fd\u4fdd\u8bc1\u4f8b\u5b50\u59cb\u7ec8\u53ef\u7528\u3002 \u5b9e\u9645\u505a\u9879\u76ee\u5f00\u53d1\u65f6\uff0c\u53ef\u80fd\u4f1a\u9762\u5bf9\u6210\u767e\u4e0a\u5343\u4e2a\u6587\u672c\u6587\u4ef6\uff0c\u5982\u679c\u6709\u51e0\u4e2a\u6587\u4ef6\u6df7\u5165\u4e86BOM\uff0c\u90a3\u4e48\u5f88\u96be\u5bdf\u89c9\uff0c\u5982\u679c\u6ca1\u6709\u5e26BOM\u7684UTF-8\u6587\u672c\u6587\u4ef6\uff0c\u53ef\u4ee5\u7528vi\u675c\u64b0\u51e0\u4e2a\uff0c\u76f8\u5173\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\n\u8bbe\u7f6eUTF-8\u7f16\u7801\uff1a\r\n:set fileencoding=utf-8\r\n\r\n\u6dfb\u52a0BOM\uff1a\r\n:set bomb\r\n\r\n\u5220\u9664BOM\uff1a\r\n:set nobomb\r\n\r\n\u67e5\u8be2BOM\uff1a\r\n:set bomb?\r\n\r\n\u5982\u4f55\u68c0\u6d4bUTF-8\u7f16\u7801\u4e2d\u7684BOM\u5462\uff1f\r\nshell&gt; grep -r -I -l $\u2019^\\xEF\\xBB\\xBF\u2019 \/path\r\n\r\n\u5982\u4f55\u5220\u9664UTF-8\u7f16\u7801\u4e2d\u7684BOM\u5462\uff1f\r\nshell&gt; grep -r -I -l $\u2019^\\xEF\\xBB\\xBF\u2019 \/path | xargs sed -i \u2018s\/^\\xEF\\xBB\\xBF\/\/;q\u2019\r\n<\/pre>\n<p>\u63a8\u8350\uff1a\u5982\u679c\u4f60\u4f7f\u7528SVN\u7684\u8bdd\uff0c\u53ef\u4ee5\u5728pre-commit\u94a9\u5b50\u91cc\u52a0\u4e0a\u76f8\u5173\u4ee3\u7801\u7528\u4ee5\u675c\u7eddBOM\u3002<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\n#!\/bin\/bashREPOS=&quot;$1&quot;TXN=&quot;$2&quot;SVNLOOK=\/usr\/bin\/svnlookfor FILE in $($SVNLOOK changed -t &quot;$TXN&quot; &quot;$REPOS&quot; | awk \u2018\/^&#x5B;AU]\/ {print $NF}\u2019); do    if $SVNLOOK cat -t &quot;$TXN&quot; &quot;$REPOS&quot; &quot;$FILE&quot; | grep -q $\u2019^\\xEF\\xBB\\xBF\u2019; then        echo &quot;Byte Order Mark be found in $FILE&quot; 1&gt;&amp;2        exit 1    fidone\r\n<\/pre>\n<p>\u672c\u6587\u7528\u5230\u4e86\u5f88\u591ashell\u547d\u4ee4\uff0c\u7bc7\u5e45\u6240\u9650\uff0c\u6055\u4e0d\u8be6\u8ff0\uff0c\u5982\u679c\u6709\u4e0d\u660e\u767d\u7684\u5c31\u8bf7\u81ea\u5df1\u641c\u7d22\u5427\u3002<\/p>\n<p>2014-09-09\u66f4\u65b0\uff1a<\/p>\n<p>\u5728linux\u7528shell\u751f\u6210\u4e86\u4e00\u4e2acsv\u6587\u4ef6\uff0c\u7ed3\u679c\u62ff\u5230windows\u4e0b\u9762\u7528excel\u6253\u5f00\u4e71\u7801\uff0c\u641e\u4e86\u51e0\u4e2a\u5c0f\u65f6\u540e\u53d1\u73b0\u4e5f\u662f\u6ca1\u6709BOM\u5934\u7684\u539f\u56e0\u3002<br \/>\n\u7528sed\u547d\u4ee4\u6dfb\u52a0BOM\u5934\uff1a<br \/>\nsed -i -e &#8216;1s\/^\/\\xEF\\xBB\\xBF\/&#8217; \/home\/kyle\/shell\/import_vtalk.csv<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u6240\u8c13BOM\uff0c\u5168\u79f0\u662fByte Order Mark\uff0c\u5b83\u662f\u4e00\u4e2aUnicode\u5b57\u7b26\uff0c\u901a\u5e38\u51fa\u73b0\u5728\u6587\u672c\u7684\u5f00\u5934\uff0c\u7528\u6765\u6807\u8bc6\u5b57 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-1880","post","type-post","status-publish","format-standard","hentry","category-skill"],"_links":{"self":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/1880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/comments?post=1880"}],"version-history":[{"count":3,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/1880\/revisions"}],"predecessor-version":[{"id":5724,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/1880\/revisions\/5724"}],"wp:attachment":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/media?parent=1880"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/categories?post=1880"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/tags?post=1880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}