【机器学习】xgboost系列丨xgboost原理及公式推导
机器学习初学者
共 2394字,需浏览 5分钟
·
2021-01-05 05:42
![](https://filescdn.proginn.com/63326fc81c225a7aa6040c7cffb5b06c/bab51ac716981aae93445a373fcd9201.webp)
建树过程中如何选择使用哪个特征哪个值来进行分裂? 什么时候停止分裂? 如何计算叶节点的权值? 建完了第一棵树之后如何建第二棵树? 为防止过拟合,XGB做了哪些改进
树的集成
![](https://filescdn.proginn.com/15697411b44d405f1aee598d90f3374d/6dcbdadb10401dc29d67c48ea62ef74e.webp)
![](https://filescdn.proginn.com/bf2fd07ca8094d9ab6a480a7d233f6b0/29ff53075eb866e2f34dcfbe75851eb3.webp)
![](https://filescdn.proginn.com/8528d45ca54ea8230d4eddf33ffb934a/d12cdb5d4a97dae8b103228248d863d7.webp)
![](https://filescdn.proginn.com/cd797a233dfb82a3b3e30d23d20ee62b/dbcd86fb0936a7969729ac544cfa5218.webp)
为子模型的预测函数,每个
即是一棵树。
![](https://filescdn.proginn.com/a705a54b2793468be836ca8637e482f6/d3a2cc82cd919692b733cf4dd39444ea.webp)
![](https://filescdn.proginn.com/65bee6fa0bc630d7eca150246bcbd30d/efab76a6e62ee0de64638f9114ec896f.webp)
![](https://filescdn.proginn.com/5ca0fd4e6502d271c6b2199aba21b959/3a06e3493fe126a17e6ea702bbe48a53.webp)
![](https://filescdn.proginn.com/b763bd4096db20b4984129423e2733fc/7a4c116027521082d71885870600ef7e.webp)
![](https://filescdn.proginn.com/681cd7205ea7e940c850763abff9c950/9c296cb1c1b49bec4dd4a8b78404ab05.webp)
![](https://filescdn.proginn.com/681cd7205ea7e940c850763abff9c950/9c296cb1c1b49bec4dd4a8b78404ab05.webp)
![](https://filescdn.proginn.com/1f539ecbfadf94fa253e9aeaa9e4a158/6a59afdb2597a5014af8200a35022287.webp)
![](https://filescdn.proginn.com/8a97deb57e357870f6eeef83bf964116/d18addca544e4a8e692525746d427386.webp)
![](https://filescdn.proginn.com/72b2848e278bad4f2f3da5e94824642c/f1c9bb2d37db4bb0b2310b0e37cb0bab.webp)
![](https://filescdn.proginn.com/e85d6bb716d12deec25faec36821cdcf/815e11c80d6386a3b1a36ae15404dc27.webp)
![](https://filescdn.proginn.com/72b2848e278bad4f2f3da5e94824642c/f1c9bb2d37db4bb0b2310b0e37cb0bab.webp)
![](https://filescdn.proginn.com/8dd4c945eefb09f888692a38c5c27f94/76ee99f3365ab7e1c7ff066e3a5489c3.webp)
![](https://filescdn.proginn.com/1e66ac2d7cdc73b9b1547d5f028c73a0/5a4ab9dd3bc91e96971fc2a3e3f5bc20.webp)
![](https://filescdn.proginn.com/a4f2c4da5b60579bd78240d076ca589c/73e7a03341ba76b1062cae37149d34c4.webp)
![](https://filescdn.proginn.com/a4f2c4da5b60579bd78240d076ca589c/73e7a03341ba76b1062cae37149d34c4.webp)
为第t棵树对第i个样本做出的预测。我们每次添加新树的时候,要优化的目标函数为上一个树产生的损失。
![](https://filescdn.proginn.com/54cce267748d2daecbfc4b4de3b718e0/879e3e0b7b8bdf464fbc3ced6b3d9057.webp)
![](https://filescdn.proginn.com/c2546def04f0126050ca0d7f71285bfb/ef0f62330242b0301f215cdfd1c2903d.webp)
![](https://filescdn.proginn.com/40cdea63ac89b8ece5295c689e41ed54/5479444b83000f5a3f1120546ad84924.webp)
![](https://filescdn.proginn.com/fa4e53055c75530c0d88e854f2ae7d86/fef030b8b6c5e97bf55af54978d799b9.webp)
![](https://filescdn.proginn.com/1616b87ef7dde62958618c015761c22d/384f752f1ecc119080817c5347e0e8cc.webp)
![](https://filescdn.proginn.com/65ea4b8f5fb009398768ea897e1c4a35/d791730d1c44b1e5f579b18331cd3e02.webp)
![](https://filescdn.proginn.com/ed80dab7dccdca8a690d72433e863a9a/c327d0526429732368aeda1e0d184f1b.webp)
![](https://filescdn.proginn.com/c5956c288c69df0028b582d445c74134/52b5e8ff693301643eec4ed23643efb2.webp)
![](https://filescdn.proginn.com/2a1296332b87259df55569cba0c80708/93fbde0567a3686881664f6c1b6dcfd7.webp)
![](https://filescdn.proginn.com/d3cd352dc46a3e72fbf0747fd1d94102/43e3b89d925a22a0d702fc936f60ee94.webp)
![](https://filescdn.proginn.com/5a7d390eb5fcca08a5b281ca1206e2e9/119c43100aee6a4a34982c9cd1bb8c2e.webp)
![](https://filescdn.proginn.com/70312d58fcfa18dbf545164b7013ab90/373d9eed0be8a3302890bed8b9537426.webp)
![](https://filescdn.proginn.com/c6387f2a7ee737eff9e64cee6822c8c8/4dbb85cb8ac4b084be0c8a166a9533ca.webp)
![](https://filescdn.proginn.com/7bcd8cd5788da0a6e88442ff3fdb8a30/1d5373e1719a2645a71cca7d80bfd0e7.webp)
![](https://filescdn.proginn.com/43edfc0103e30b6b792555c551277cd4/b77c725d131488c8629029bbce935edf.webp)
![](https://filescdn.proginn.com/8ff5e79f9003ae358212f19730ce83bd/989a1043fc4aa39212e95f169b686b43.webp)
![](https://filescdn.proginn.com/a03c833e2ccb85b6f36776c4aba7c2a8/16c2eb99f4ef5911d748c32c43fe1c6a.webp)
![](https://filescdn.proginn.com/b5def6f29eead3bb02aecd086b5cd5d8/91c9cc0cfa1b7fc2ce2d20476b837a4b.webp)
![](https://filescdn.proginn.com/b5def6f29eead3bb02aecd086b5cd5d8/91c9cc0cfa1b7fc2ce2d20476b837a4b.webp)
![](https://filescdn.proginn.com/8ff5e79f9003ae358212f19730ce83bd/989a1043fc4aa39212e95f169b686b43.webp)
![](https://filescdn.proginn.com/a03c833e2ccb85b6f36776c4aba7c2a8/16c2eb99f4ef5911d748c32c43fe1c6a.webp)
![](https://filescdn.proginn.com/5ac1e620f02344cafff9d50c8c447f7c/2069ae66a26eaf0ca669d62fd5991b49.webp)
![](https://filescdn.proginn.com/d349b00fcf460f641766f81bf62c4e54/00d1d2c091abd7f1da73f8b5166cbfc9.webp)
![](https://filescdn.proginn.com/142c10aa7dbc7d7ae5727d51d86e695a/f203ce462b23bfd477bc45df7e015298.webp)
![](https://filescdn.proginn.com/884a65ea3fea0c30d185e5bbf481eeec/b9094f12b8e649b082f5b32135daa2a5.webp)
![](https://filescdn.proginn.com/b5def6f29eead3bb02aecd086b5cd5d8/91c9cc0cfa1b7fc2ce2d20476b837a4b.webp)
![](https://filescdn.proginn.com/ad2ee91a25843154a875ad8f57dc8fb0/6a6013d251af114c6cb7d267f879dfd4.webp)
![](https://filescdn.proginn.com/fdffbca1b8eea43c3ee095544baf7e4f/8475737b509e399a7af6762895d9ab92.webp)
![](https://filescdn.proginn.com/5ac1e620f02344cafff9d50c8c447f7c/2069ae66a26eaf0ca669d62fd5991b49.webp)
![](https://filescdn.proginn.com/3ab6a9f8d8eed32de527c8778ce284b6/b1e1cbf2be7c9fe662acba1514ae1cca.webp)
![](https://filescdn.proginn.com/c49c67d845b0a462314b9c605861732a/0fb77963ed94b02c50646fb4849cad1f.webp)
![](https://filescdn.proginn.com/0f6b38669292f99272d83af720e9134d/529c7ed4b47bfee3cdc8b1fbdd0b8d05.webp)
![](https://filescdn.proginn.com/3a1b179d4c228af1d7a80dfb76822cce/ecc9fbc2fd0078a1bf219490016b11ae.webp)
![](https://filescdn.proginn.com/0b84b939dd597b508570c816d02937b7/b415c34ba61058e3571aaeda7e60dd65.webp)
![](https://filescdn.proginn.com/80374b24cb64bf70392b33263544b5ac/5532172ae5ee581620bf41ef6d9c3e03.webp)
![](https://filescdn.proginn.com/ea00f18cfcdae4690c2b2c609ca636c2/635f6356987dd78a0c766f2f058abdef.webp)
![](https://filescdn.proginn.com/1768d4ed63c5261ee47142e63b31f78c/ce212542c2e4bcdcf68bf9d155638250.webp)
![](https://filescdn.proginn.com/3715d870e45d9a7c7f8e7f2c332ef652/68dfbf8549ae629eaf90d4b3dd3b26cd.webp)
叶节点权重
![](https://filescdn.proginn.com/5016c2e7bda58fec675bee6b348d879b/af24398d6a76a7d0fca1c8a8bf067441.webp)
![](https://filescdn.proginn.com/1d276eb0db35686c75c3373d5d1bf11f/832e260cacee20e145d6f44ecf36ac44.webp)
![](https://filescdn.proginn.com/2d5ff727c02619584cd5f63df73d0823/42e00db9f93063fef1631a931d7abe7b.webp)
![](https://filescdn.proginn.com/5ac1e620f02344cafff9d50c8c447f7c/2069ae66a26eaf0ca669d62fd5991b49.webp)
![](https://filescdn.proginn.com/f42b5017ad49223f12f763bb883929b2/9131627fc0b9934cc8d6616428f74256.webp)
![](https://filescdn.proginn.com/c1c435785cafe478251d882966a89a92/39c73d5f7fd7af5c1c547be59e7b39ef.webp)
![](https://filescdn.proginn.com/db2d4b97cb25e3f05b42c1a879a4a05c/52f71b10b8fefd03c1b4a5cb5d51aa3f.webp)
![](https://filescdn.proginn.com/674d507f15bdd5706b194abc1e6b410c/ddb02c96974e797fc6cd292ecb632f68.webp)
![](https://filescdn.proginn.com/db2d4b97cb25e3f05b42c1a879a4a05c/52f71b10b8fefd03c1b4a5cb5d51aa3f.webp)
![](https://filescdn.proginn.com/9e274e083d5f5d026d1ec93529955095/2888ecebc085cd1c76f1204bad70099a.webp)
![](https://filescdn.proginn.com/99324b56f048cf27bd5ff0e3a17c1e5d/6445a70734779cf4d52b959609d33e8d.webp)
![](https://filescdn.proginn.com/9922d5b7167c71998401e03b74749980/90627ec64ceeea8ab65ad724751da638.webp)
![](https://filescdn.proginn.com/fff082bb08560e8d57eff146dd0ef830/72871a3979748ae6f8aeaed96598b856.webp)
![](https://filescdn.proginn.com/f42b5017ad49223f12f763bb883929b2/9131627fc0b9934cc8d6616428f74256.webp)
![](https://filescdn.proginn.com/64943805f3b5f9adad823ed43ba286f1/3a5eeb1bfedf9ed9c03551a9074cbd3b.webp)
分裂准则
![](https://filescdn.proginn.com/cb68d9f7cc24d611528b149454e8f5ee/95e4043a43adaab7db89384bf5a7ab64.webp)
![](https://filescdn.proginn.com/570916604fbe2c7d61de49c16e5fc837/5f66bf3348da3f809f242e18c8a832cc.webp)
![](https://filescdn.proginn.com/71b3209d2f49c8a615d9231bfcea807f/01effaf8a56a66ff1cda5e9a225d4cbd.webp)
![](https://filescdn.proginn.com/ad6699479bef722bd8e16caa1413dab0/0b3675255c67a33bbf4bd013d5a1b43f.webp)
![](https://filescdn.proginn.com/570916604fbe2c7d61de49c16e5fc837/5f66bf3348da3f809f242e18c8a832cc.webp)
![](https://filescdn.proginn.com/5298ea5c09bc526874da6af34aba606d/3f1c9c61a7b13f3ca17ce131501d7be9.webp)
![](https://filescdn.proginn.com/42f692357602d605ea5635439da190f4/e47e0891380c0a9511219e32b80b036c.webp)
![](https://filescdn.proginn.com/7d3d8b0ae02edc095c2f85ae50457d72/58babb1727a0dc77993cca0422ae49ce.webp)
![](https://filescdn.proginn.com/b2932829a8ce63e54c5cda7c9a21e0d4/e7afa11e92bb9838084a3d6f583d15b6.webp)
![](https://filescdn.proginn.com/7ed80453f4df4e169548ef72dfeac057/f455e1e4e0d75279f6b63b9a73d9a181.webp)
![](https://filescdn.proginn.com/8e053dc12a68838c78dcf9a767849413/f36da2caa4a2c48959c8e4f3b715e3c6.webp)
![](https://filescdn.proginn.com/71b3209d2f49c8a615d9231bfcea807f/01effaf8a56a66ff1cda5e9a225d4cbd.webp)
![](https://filescdn.proginn.com/71b3209d2f49c8a615d9231bfcea807f/01effaf8a56a66ff1cda5e9a225d4cbd.webp)
损失函数计算
![](https://filescdn.proginn.com/ff798530a5734391d08b038b58455df8/2c05308263b7c4ada3d66cb843d4b414.webp)
![](https://filescdn.proginn.com/32b258c02b4b34700e310b4cc18bb636/f6a631e333004fcce6bd97ad82f27a3e.webp)
![](https://filescdn.proginn.com/f01ba3d6ff6eb147a98d8a2373bcb506/7d9558609ab8df9e04a493e469317f31.webp)
![](https://filescdn.proginn.com/562b2cd77d87db9c672f15537e025312/43935e423f519a97e52668fef91130e5.webp)
![](https://filescdn.proginn.com/7b555dbf91f4851b22d21d80dc6e2192/8e8dd8eda69a7571bd08f54cc8b163ad.webp)
![](https://filescdn.proginn.com/498b04a524a3af0e503d12ce0ec25c7d/635e0aa393de53de053c0f406b972e16.webp)
![](https://filescdn.proginn.com/5458449b794a8b601bc59cc7293312a0/1c62d7037985fb1f108d5cd949282ccc.webp)
![](https://filescdn.proginn.com/53928dde04876ff75805fb0b110b35ef/106d202e47b2c115cf494387d38e53e7.webp)
往期精彩回顾
获取本站知识星球优惠券,复制链接直接打开:
https://t.zsxq.com/qFiUFMV
本站qq群704220115。
加入微信群请扫码:
评论