• <input id="4g248"><object id="4g248"></object></input>
    立即打開
    創造阿爾法狗的公司,如今要解開生物學最大秘密

    創造阿爾法狗的公司,如今要解開生物學最大秘密

    Jeremy Kahn 2020年12月22日
    DeepMind首創的新方法在抗擊新冠病毒的斗爭中已經取得成果。本文將闡述這家以游戲知名的公司如何解開生物學最大秘密的故事。

    計算機生成與新冠病毒相關的蛋白質ORF8圖像。圖像由DeepMind開發的人工智能系統支持繪制。圖片來源:COURTESY OF DEEPMIND

    2016年3月13日深夜,氣溫相當寒冷,兩名男子頭戴羊毛帽,身穿厚厚的外套,并肩走過韓國首爾市中心擁擠的街道。二人熱烈地交談,似乎完全忽視了周圍餃子館和燒烤店霓虹燈的誘惑。他們此行韓國肩負重任,多年的努力終于能夠看到結果。最棒的是,他們剛剛成功了。

    這次散步是為了慶祝。他們取得的成就將進一步鞏固他們在計算機史上的地位。在古老的戰略游戲圍棋領域里,他們開發的人工智能軟件已經充分掌握了個中奧秘,而且輕松擊敗了全球頂尖選手李世石。如今,兩人開始討論下一個目標,身后跟蹤的紀錄片攝制組捕捉到了當時的談話。

    “告訴你,我們可以解決蛋白質折疊問題?!钡旅姿?哈薩比斯對同伴大衛?西爾弗說?!澳遣攀谴蟪删?。我相信現在能夠去做了。以前我只是想過,現在肯定可以做成?!惫_比斯是總部位于倫敦的人工智能公司DeepMind的聯合創始人及首席執行官,正是該公司開發出了AlphaGo(阿爾法狗)。西爾弗則是DeepMind的計算機科學家,負責領導AlphaGo團隊。

    四年后,DeepMind實現了當年哈薩比斯在首爾散步時的設想。公司開發出了人工智能系統,能夠根據基因序列來預測蛋白質的復雜形狀,精確到單個原子寬度??恐@項成就,DeepMind完成了需要近50年才能完成的科學探索。1972年,化學家克里斯蒂安?安芬森在諾貝爾獎獲獎演說中提出,只有DNA才可以完全決定蛋白質的最終結構。這是驚人的猜想。當時連一個基因組都未完成測序。安芬森的理論開創了計算生物學的分支,目標是用復雜的數學模擬蛋白質結構,而不是實驗。

    DeepMind在圍棋方面取得的成就確實很重要,但在圍棋和計算機科學這兩個相對偏僻的領域之外,幾乎沒有產生什么具體影響。解決蛋白質折疊問題則完全不同,對大多數人來說都有變革意義。蛋白質是生命的基本組成部分,也是大多數生物過程背后的運行機制。如果能夠預測蛋白質的結構,將徹底改變人們對疾病的理解,還可以為癌癥到老年癡呆癥等各種疾病開發全新也更具針對性的藥物。新藥上市時間有望加快,藥物研發成本減少數年時間,成本也節約數億美元,還可能會拯救很多生命。

    DeepMind的聯合創始人及首席執行官德米斯?哈薩比斯。他早年癡迷國際象棋和電子游戲設計,后來對開發人工智能系統產生興趣。圖片來源:Courtesy of DeepMind

    DeepMind首創的新方法在抗擊SARS-CoV-2(也就是新冠病毒)的斗爭中已經取得成果。以下是以游戲知名的公司如何揭開生物學最大秘密的故事。

    形狀莫測的積木

    “蛋白質是細胞的主要機器?!奔又荽髮W伯克利分校的生物工程教授伊恩?霍姆斯表示。蛋白質的結構和形狀對其工作方式至關重要,構成蛋白質分子晶格的小“口袋”是發生各種化學反應的地方。如果能夠找到某種化學物質與其中一個口袋結合,這種物質就可以作為藥物阻止或加速生物過程。生物工程師還能夠創造出自然界中從未出現的全新蛋白質,而且具有獨特的療效?!叭绻覀兛梢岳玫鞍踪|的力量,合理地設計用途,就能夠制造出神奇的自我組裝機器,發揮一些作用?!被裟匪拐f。

    但為了確保蛋白質達到想要的效果,把握其形狀很重要。

    蛋白質由氨基酸鏈組成,常被比作細繩上的珠子。至于珠子按照什么順序穿起來,信息都存儲在DNA里。但是,根據簡單的基因指令很難預測完整的鏈條會形成多復雜的物理形狀。氨基酸鏈根據分子間吸引和排斥的電化學規則折疊成某種結構。形狀常常類似繩索和絲帶纏繞而成的抽象雕塑:褶皺的帶狀物加上莫比烏斯帶,就像卷曲環狀的螺旋。20世紀60年代,物理學家和分子生物學家塞勒斯?列文塔爾發現,一種蛋白質的形狀有太多可能性。如果想通過隨機嘗試組合找出蛋白質的準確結構,花的時間比已知宇宙的年齡還長。而且,幾毫秒內蛋白質就會完成折疊。該觀察被稱為列文塔爾悖論。

    到目前為止,只有通過所謂X射線晶體衍射才可以接近準確了解蛋白質的結構。顧名思義,首先需要將含有數百萬蛋白質的溶液轉化為晶體,本身就是很復雜的化學過程。然后,X射線發射到晶體上,科學家從獲得的衍射圖逆向工作,從而建立蛋白質圖像。而且,還不是隨便什么X射線都可以。要想獲得很多蛋白質的結構,要由圓形的,大小堪比體育場的同步加速器發射X射線。

    過程既昂貴又耗時。根據多倫多大學(University of Toronto)的研究人員估計,用X射線晶體衍射法測定單個蛋白質的結構需要約12個月,花費約12萬美元。已知的蛋白質超過2億種,每年大約能夠發現3000萬種,但其中只有不到20萬種蛋白質通過X射線晶體衍射或其他實驗方法繪制出了結構圖?!叭祟惖臒o知程度正在迅速增長?!庇嬎阄锢韺W家約翰?喬普說,現在他擔任DeepMind的高級研究員,負責領導蛋白質折疊團隊。

    過去50年里,自從克里斯蒂安?安芬森發表著名演講以來,科學家們一直努力使用高性能計算機上運行的復雜數學模型加速分析蛋白質結構?!盎旧暇褪菄L試在計算機里創建蛋白質的數字雙胞胎,然后嘗試操作?!瘪R里蘭大學的細胞生物學和分子遺傳學教授約翰?穆爾特說,他也是用數學算法通過DNA序列預測蛋白質結構的先驅。問題是,預測出的折疊模式經常有誤,與科學家通過X射線晶體衍射發現的結構并不一致。事實上大約10年前,很少有模型預測大蛋白質形狀時準確率可以超過三分之一。

    蛋白質折疊模擬要占用龐大的算力。2000年,研究人員創建了名叫Fold@home的“公民科學”項目,人們能夠捐出個人電腦和游戲機的閑置處理能力運行蛋白質折疊模擬。所有設備通過互聯網連接在一起,從而打造全世界最強大的虛擬超級計算機之一。大家都希望幫研究人員擺脫列文塔爾悖論,通過隨機實驗和試錯準確判斷蛋白質的結構。目前該項目仍然在進行中,已經為超過225篇論文提供了數據,研究內容是與多種疾病相關的蛋白質。

    盡管擁有強大的處理能力,Fold@home仍然深陷列文塔爾悖論,因為算法試圖搜索所有可能的排列,從而找到蛋白質結構。破解蛋白質折疊的關鍵在于跳過艱苦搜索的過程,發現蛋白質DNA序列與結構聯系的神秘模式,從而讓計算機踏上全新捷徑,直接從遺傳學領域轉到準確繪制形狀。

    嚴肅的游戲

    德米斯?哈薩比斯對蛋白質折疊的興趣始于一場游戲,他對很多事都是這樣。哈薩比斯曾經是國際象棋天才,13歲時已經成為大師,一度在同年齡里排名世界第二。他對象棋的熱愛后來轉向對兩件事感興趣:一是游戲設計,二是研究自身意識的內在機制。他高中時開始為電子游戲公司工作,在劍橋大學(University of Cambridge)學習計算機科學后,1998年創立了電腦游戲初創公司Elixir Studios。

    盡管曾經研發出兩款獲獎游戲,最終Elixir還是賣掉知識產權并關閉公司,哈薩比斯從倫敦大學學院(University College London)獲得了認知神經科學博士學位。彼時他已經開始踏上漫漫征途,后來2010年聯合創立了DeepMind。他開始研發通用人工智能軟件,不僅可以學習執行很多任務,有些甚至比人類完成得更好。哈薩比斯曾經說過,DeepMind的遠大目標是“解決智能問題,然后解決所有其他問題?!惫_比斯也曾經暗示,蛋白質折疊可能就是“其他問題”里的第一批。

    2009年,哈薩比斯在麻省理工學院(Massachusetts Institute of Technology)攻讀博士后時,聽說了一款名為Foldit的在線游戲。Foldit是由華盛頓大學(University of Washington)的研究人員設計,跟Fold@home類似,也是有關蛋白質折疊的“公民科學”項目。但Foldit并不是整合閑置的微芯片,而是利用閑置的大腦。

    Foldit是類似益智游戲的游戲,并不掌握生物學領域知識的人類玩家比賽折疊蛋白質,如果能夠得到合理的形狀就可以獲得積分。然后,研究人員分析得分最高的設計,看是否有助于破解蛋白質結構問題。游戲已經吸引成千上萬玩家,并且一些記錄案例中得到的蛋白質結構比研究蛋白質折疊的計算機算法更準確?!皬倪@個角度來看,我覺得游戲很有趣,想著能不能利用游戲的上癮性和游戲的樂趣,不僅讓人們玩得開心,也做一些對科學有用的事情?!惫_比斯說。

    Foldit能夠抓住哈薩比斯的想象力還有另一個原因。其實游戲是一種強化學習行為,特別適合訓練人工智能。軟件可以通過試驗和試錯從經驗中學習,從而更好地完成任務。在游戲里軟件能夠無休止地試驗,反復地玩,逐步改進,不對現實世界造成傷害的情況下提升技能水平,直到超過人類。游戲也有現成的方法判斷某個特定的動作或某組動作是否有效,即積分和勝利。種種指標可以提供非常明確的標準衡量表現,在現實世界很多問題里則無法如此處理?,F實世界遇到問題時,最有效的方法可能比較模糊,“獲勝”的概念也可能不適用。

    DeepMind的基礎主要是將強化學習與稱為深度學習的人工智能相結合。深度學習是基于神經網絡的人工智能,所謂神經網絡是大致基于人腦工作原理的軟件。這種情況下,軟件沒有實際的神經細胞網絡,而是一堆虛擬神經元分層排列,初始輸入層接收數據,按照權重分配后傳遞到中間層,中間層依次執行相同操作,最終傳遞到輸出層,輸出層匯總各項加權值并算出結果。網絡能夠調整各項權重,直到產生理想的結果,例如準確識別貓的照片或國際象棋獲勝。之所以被稱為“深度學習”,并不是因為產生的結果一定深刻,當然也有可能深刻,但主要原因是網絡由許多層構成,所以可以說具有深度。

    DeepMind最初成功是用“深度強化學習”創建軟件,自學玩經典的雅達利電腦游戲,如《乒乓球》(Pong)、《突圍》(Breakout)和《太空入侵者》(Space Invaders)等,而且水平超過人類。正是這一成就讓DeepMind受到谷歌(Google)等科技巨頭的關注,據報道,2014年谷歌以4億英鎊(當時超過6億美元)收購了DeepMind。之后公司主攻圍棋并開發了AlphaGo系統,2016年擊敗了李世石。DeepMind接著開發了名叫AlphaZero的更通用系統版本,幾乎能夠學會所有兩玩家回合制游戲,在這種游戲中,玩家都可以獲得充分信息(沒有機會隱藏信息,例如牌面朝下放置或隱藏位置)。去年,公司開發的系統還在高度復雜的即時戰略游戲《星際爭霸2》(Starcraft 2)中擊敗了頂尖的人類職業電競玩家。

    2016年3月15日,谷歌DeepMind挑戰賽最后一場比賽結束后,職業圍棋選手李世石(左)與德米斯?哈薩比斯握手,比賽中李世石與電腦程序AlphaGo對決。圖片來源:Jeon Heon-Kyun—Pool/Getty Images

    但哈薩比斯表示,一直認為公司在游戲方面的探索是完善人工智能系統的方式,之后能夠應用于現實世界挑戰,尤其是科學領域?!氨荣愔皇怯柧殘?,但訓練到底為了什么?最終是為了創造新知識?!彼f。

    DeepMind并非具有產品和客戶的傳統業務,本質上是推動人工智能前沿的研究實驗室。公司的很多開發方法都已經公開,供所有人使用或借鑒。不過某些方面的進步對姊妹公司谷歌也頗有幫助。

    DeepMind團隊由工程師和科學家組成,幫助谷歌將尖端的人工智能技術融入產品。DeepMind的技術已經滲透各處,從谷歌地圖(Google Maps)到數字助理,再到協助管理安卓手機電池電量的系統。谷歌為此向DeepMind支付費用,母公司Alphabet繼續承擔DeepMind帶來的額外虧損。虧損規模并不小,2018年,公司虧損4.7億英鎊(當時約合5.1億美元),這也是通過英國的商業注冊機構公司登記局(Companies House)可以查到的最新一年公開記錄。

    不過如今員工超過1000人的DeepMind,還有一整個部門只負責人工智能的科學應用。該部門的負責人為39歲的印度人普什米?科里,他加入DeepMind之前曾經在微軟從事人工智能研究。他表示,DeepMind的目標是解決“根節點”問題,這是數據科學家的慣用語,意思是希望解決能夠解鎖很多科學路徑的基礎問題。蛋白質折疊就是根節點之一,科里說。

    “蛋白質折疊的奧運會”

    1994年,當很多科學家剛開始使用復雜的計算機算法預測蛋白質折疊方式時,馬里蘭大學的生物學家墨爾特決定開辦競賽,用公正的方法評估哪種算法最好。他把比賽稱為蛋白質結構預測關鍵評估(簡稱為CASP),之后每兩年舉辦一次。

    賽事具體如下,美國國立衛生研究院資助的蛋白質結構預測中心主辦CASP,并說服從事X射線晶體衍射和其他實證研究的研究人員提供尚未公布的蛋白質結構,要求在CASP競賽結束之前不公開相關結構。然后CASP將蛋白質DNA序列發給參賽者,參賽者用算法預測蛋白質結構。CASP判斷預測與X射線晶體學家和實驗學家發現的實際結構接近程度,然后根據算法對各種蛋白質預測的平均得分排名?!拔曳Q之為蛋白質折疊界的奧運會?!惫_比斯說。2016年AlphaGo擊敗李世石后不久,DeepMind就打算贏得金牌。

    DeepMind組建了小規模精干的團隊,由六名機器學習研究人員和工程師組成?!白尅ú拧胧质俏覀兊睦砟??!惫_比斯說。公司里并不缺乏人才?!扒拔锢韺W家、前生物學家,大家都四處閑逛?!惫_比斯有點啼笑皆非?!八麄冇肋h不知道之前的專業知識什么時候可以突然發揮作用?!弊詈髨F隊成員增加到20人左右。

    不過,DeepMind還是認為團隊里至少要有一位真正的蛋白質折疊專家,后來選中了約翰?喬普。35歲的喬普像個大男孩,瘦得皮包骨,一頭蓬亂斜梳的棕色頭發,有點像20世紀90年代末高中車庫樂隊的低音吉他手。他在劍橋大學獲得理論凝聚態物理碩士學位,之后在紐約由對沖基金億萬富翁大衛?肖創立的獨立研究實驗室D.E.Shaw Research工作。實驗室專門研究計算生物學,包括蛋白質模擬。后來喬普在芝加哥大學獲得了計算生物物理學博士學位,導師為卡爾?弗里德和托賓?索斯尼克,兩位科學家皆因推動蛋白質折疊模型進步出名?!拔以浡犝fDeepMind對解決蛋白質結構有興趣?!彼f。于是他申請并順利加入。

    哈薩比斯和DeepMind團隊的第一直覺是,蛋白質折疊能夠用與圍棋完全相同的方式解決,即深度強化學習。事實證明存在問題。首先,蛋白質折疊結構的可能性比圍棋的步數還要多。更重要的是,DeepMind讓工智能系統AlphaGo與自己對弈就可以掌握圍棋的玩法?!八钥杀刃圆⒉桓?,因為蛋白質折疊不是雙人游戲?!惫_比斯說,“有點違背自然?!?/p>

    計算物理學家約翰?喬普如今負責DeepMind的蛋白質折疊團隊。喬普說,團隊面臨的挑戰不僅是在競爭中領先:“我們想打造對生物學家很重要的系統?!眻D片來源:Courtesy of DeepMind

    DeepMind很快發現,如果使用所謂監督式深度學習的人工智能培訓方法,就能夠更簡便地取得進步。這是大多數商業應用里使用的人工智能,神經網絡通過一組既定數據輸入和相應輸出,可以學習如何將給定的輸入與給定輸出相匹配。具體到蛋白質結構,DeepMind已經掌握約170000個蛋白質結構,能夠作為訓練數據。蛋白質數據庫(PDB)是已知三維蛋白質形狀及遺傳序列的公共存儲庫,可以公開查詢相關結構。

    一些生物學家已經使用監督式深度學習預測蛋白質如何折疊。但此類人工智能系統表現最佳的正確率也只有50%,對生物學家或醫學研究人員沒有什么幫助,尤其是對結構未知的蛋白質,因為無法確定某次特定預測是否正確。

    有種技術很有希望,其理念是基于蛋白質的進化史劃分為不同的家族。各種家族里可能在一個DNA序列中找到相距遙遠但似乎會同時突變的氨基酸對。此類所謂“共同進化”的現象很有幫助,因為共同進化的蛋白質很可能在蛋白質折疊結構中有聯系。位于芝加哥的豐田技術研究所(Toyota Technological Institute)的科學家徐金波(音譯)率先利用深入學習共同進化數據預測氨基酸聯系。這種方法有點像是在連接點游戲里尋找點??茖W家仍然要用其他軟件找出點之間的線,過程中經常出錯。有時候連點都找不準。

    在2018年的CASP競賽中,DeepMind應用了共同進化和預測聯系的基本思想,但增加了兩個重要的轉折點。首先,系統沒有試圖確定兩個氨基酸是否有聯系,也就是二進制輸出(即兩個氨基酸可能有聯系,也可能沒有聯系),而是決定讓算法預測蛋白質里所有氨基酸對之間的距離。

    在多數分子生物學家看來,這種方法似乎違反直覺,不過值得稱贊的是,徐金波也獨立提出了類似方法。畢竟,聯系才是最重要的。對于DeepMind的深度學習專家來說,很明顯距離是讓神經網絡發揮作用更好的指標,科里表示?!斑@只是深度學習的基礎部分,如果與決策相關存在不確定性,最好是讓神經網絡整合不確定性,并決定如何應對?!彼f。與聯系不一樣,距離包含了神經網絡可調整和使用的豐富信息。

    DeepMind另一項讓人意外之處是引入第二個神經網絡,用于預測氨基酸對之間的角度。有了距離和角度兩個因素,DeepMind的算法就能夠算出蛋白質結構的大致輪廓。然后,系統使用另一種非人工智能算法改進結構。DeepMind將相關組件整合到名為AlphaFold的系統中,橫掃了2018年CASP(又稱為第13屆CASP,因為是兩年一度比賽舉辦第13次。)比賽里結構最復雜的43種蛋白質中,AlphaFold在25種蛋白質中得分最高。第二名僅在三種蛋白質里得到高分。研究結果震驚了全行業。如果說之前還有人懷疑深度學習究竟是不是解決蛋白質折疊問題最有希望的方法,AlphaFold讓所有人再無疑問。

    回到白板

    盡管如此,DeepMind還遠沒有達到哈薩比斯的目標,即完全解決蛋白質折疊問題。AlphaFold準確率只有一半,第13屆CASP的104個蛋白質中,準確度可以達到X射線晶體衍射水平的只有三個?!拔覀儾恢幌朐贑ASP競賽中奪魁,而是想真正解決問題。我們想打造對生物學家很重要的系統?!眴唐照f。

    2018年CASP的結果公布后不久,DeepMind就開始加倍努力。喬普負責擴大的團隊。團隊并未簡單地在AlphaFold基礎上改進,而是返回原點,集思廣益尋找完全不同的想法,他們希望新創意能夠幫軟件將精確度提升到更接近X射線晶體衍射級別。

    喬普表示,接下來是整個項目中最可怕也最令人沮喪的時期之一,因為什么辦法都沒有?!拔覀兓巳齻€月,結果都達不到第13屆CASP的水平,開始真正感覺到恐慌?!彼f。不過當時研究人員的嘗試出現了一些改進,沒到6個月系統已經比最初的AlphaFold有了明顯改進。之后兩年里一直延續該模式,喬普說。先是三個月一無所獲,接下來三個月快速發展,接著又是平臺期。

    哈薩比斯說,DeepMind以前的項目也出現過類似模式,包括圍棋項目,還有復雜的即時戰略游戲《星際爭霸2》項目。他說,公司克服問題的管理策略就是交替采取兩種不同的工作方式。第一種哈薩比斯稱之為“攻擊模式”,盡可能推動團隊,追求當前系統可以達到的極致表現。然后,全力以赴努力的效果似乎耗盡時,他就開始轉向所謂的“創新模式”。期間哈薩比斯不再對團隊施加壓力,容忍甚至期待出現暫時性的后退,從而為研究人員和工程師提供修補新想法和嘗試新手段的空間。他說:“要鼓勵人們提出盡可能多的瘋狂想法,還要頭腦風暴?!痹撃J酵ǔD軌蛲苿有阅艹霈F新飛躍,讓團隊切換回攻擊模式。

    生日大禮

    2019年11月21日,DeepMind蛋白質折疊團隊的研究員凱薩倫?圖雅蘇那科年滿30歲。這一天也會因為另一個原因值得紀念。圖雅蘇那科擁有牛津大學(University of Oxford)計算生物學博士學位,在團隊里負責為蛋白質折疊人工智能開發新測試集,新款人工智能叫AlphaFold 2,是DeepMind為2020年的CASP競賽新開發的系統。那天早上她打開辦公電腦時,收到系統對一批大約50個蛋白質序列預測的評估,所有序列均為最近才添加到蛋白質數據庫中。她愣了一下,然后大吃一驚。AlphaFold 2確實一直在改進,但對該組蛋白質的預測結果驚人地準確。系統對好幾個蛋白質結構結構預測誤差在1.5埃以內,埃的距離單位相當于十分之一納米,或大約一個原子的寬度。

    DeepMind的科學家凱薩倫?圖雅蘇那科幫助公司在蛋白質折疊研究方面取得了進展。圖片來源:Courtesy of DeepMind

    自稱“團隊悲觀主義者”的圖雅蘇那科說,第一反應并不是高興而是有點想吐?!拔耶敃r很害怕?!彼f。結果實在太好,她以為是自己犯了錯,可能準備測試集時無意中把人工智能在訓練數據里見過的幾個蛋白質加了進來。如此一來AlphaFold 2基本上就可以作弊,輕易預測出準確的結構。圖雅蘇那科回憶說,當時坐在DeepMind自助餐廳俯瞰倫敦的圣潘克拉斯車站(St. Pancras Station),一杯接一杯地喝茶努力平復心情。隨后,她和其他團隊成員花了一整天,直到深夜才下班,之后幾天也是如此,他們坐在工作站旁埋頭梳理AlphaFold 2的訓練數據,希望找出錯誤所在。

    然而一個錯誤也沒有。事實是,新系統在預測表現方面實現了巨大飛躍。AlphaFold 2與之前版本完全不同。人工智能不再只是各成分組合,一個用來預測氨基酸之間的距離,另一個預測角度,然后用第三個軟件聯系起來?,F在的人工智能用單一的神經網絡直接從DNA序列進行推理。雖然系統仍然接受進化信息,從而確定研究的蛋白質是否與以前見過的蛋白質有共同的祖先,并仔細檢查目標蛋白質的DNA序列與其他已知序列之間的一致性,但不再需要哪些氨基酸對共同進化的明確數據?!拔覀儾⑽刺峁└嘈畔?,反而減少了信息?!眴唐照f。系統可以自由地得出見解,即祖先何時可能決定蛋白質的部分形狀,以及何時可能徹底偏離。換句話說,系統根據經驗培養出直覺,就像老練的人類科學家一樣。

    新系統的核心是“注意力”機制,顧名思義,注意力是讓深度學習系統專注于某組輸入,并對相關輸入加大權重。舉例來說,在識別貓的系統里,系統可能學會注意耳朵的形狀,也會學習在鼻子附近尋找胡須。喬普比較了AlphaFold 2的功能與玩拼圖游戲,過程中“能夠將某些部分拼湊在一起而且非常確定,得到不同的本地解決方案,然后想辦法將相關問題連接起來?!眴唐照f,神經網絡的中層已經學會根據對DNA序列的分析推理幾何和空間排列,以及氨基酸對如何連接。

    DeepMind曾經在128個“張量處理核心”上訓練AlphaFold 2,張量處理核心是在16塊專門用于深度學習的計算機芯片上創建的數字運算大腦,芯片由谷歌設計并在數據中心使用,公司稱連續運行了數周。(128個專用的人工智能核心大約相當于100到200塊強大的圖形處理芯片,可以在Xbox或PlayStation上呈現極其炫目的動畫效果。)公司表示,經過訓練的系統提取DNA序列后“幾天內”就能夠完成整個結構預測。

    AlphaFold 2與前一代相比有個優勢,就是提供可信程度,即系統對結構里每種氨基酸的預測都有信心分數。如果說AlphaFold 2可以切實幫到生物學家和醫學研究人員,這項指標至關重要,因為研究者需要清楚何時能夠合理依賴模型,以及何時需要更加謹慎。

    盡管測試結果驚人,DeepMind仍然不能確定AlphaFold 2的預測效果。新冠病毒來襲時,公司才得到重要的線索。今年3月,AlphaFold 2可以預測出六種與SARS-CoV-2(引發疫情的病毒)相關但未被研究的蛋白質結構,后來科學家使用所謂低溫電子顯微鏡的經驗方法證實了其中一種。由此能夠充分看出AlphaFold 2對現實世界的影響力。

    驚人的結果

    CASP比賽在5月到8月之間舉行。蛋白質結構預測中心發布多批目標蛋白質,之后參賽方提交結構預測進行評估。今年比賽排名于11月30日公布。

    每次預測均可以得到“全球距離測試總分”,簡稱GDT的指標評分,該指標實際上看預測結果與通過實證方法(如X射線晶體衍射或電子顯微鏡)得到的結構接近程度,單位為埃。CASP的主席穆爾特表示,滿分是100分,如果得分能夠達到90分或以上,說明與實證方法相當。根據CASP組織者判斷的結構難度,蛋白質也會劃分不同的組。

    穆爾特看到AlphaFold 2的結果時簡直不敢相信。他就像幾個月前的圖雅蘇那科一樣,剛開始的想法是出錯了。也許比賽中一些蛋白質序列以前發表過?又或者DeepMind也許設法獲得了未發布數據的緩存?

    T1042的計算機生成圖像,T1042是感染細菌病毒里的部分蛋白質。2020年CASP競賽中,DeepMind的AlphaFold 2準確預測了該蛋白質的結構,這是人工智能在生物學和醫學研究應用方面的重大突破。圖片來源:Courtesy of DeepMind

    T1037的計算機生成圖像,T1037是感染細菌病毒里的部分蛋白質。2020年CASP競賽中,DeepMind的AlphaFold 2成功地預測了T1037的結構。圖片來源:Courtesy of DeepMind

    為了核實,他請位于德國圖賓的根馬克斯?普朗克發展生物學研究所(Max Planck Institute for Developmental Biology)的蛋白質進化系主任安德烈?盧帕斯幫忙驗證。盧帕斯讓AlphaFold 2預測一個自己確信沒有見過的結構,因為盧帕斯利用X射線結晶衍射從未成功觀測到該蛋白質的關鍵部分。近十年來,盧帕斯一直因為該部分缺失而傷腦筋,但就是觀測不到準確的形狀。盧帕斯說,利用AlphaFold的預測后,他重新查看X射線數據?!皼]到半小時就得出了正確結構?!彼f,“太令人吃驚了!”

    2018年DeepMind在CASP中獲得成功以來,諸多學術研究人員紛紛涌向深度學習技術。結果,該領域其他方面的表現都有所提高。在中等難度目標方面,其他競爭對手的平均最佳預測GDT得分為75,比兩年前提高了10分。不過還是完全追不上AlphaFold 2,因為該系統預測蛋白質結構平均得分高達92,就算面對最復雜的蛋白質平均得分也有87。穆爾特表示AlphaFold 2的預測“與實證方法不相上下”,比如X射線晶體衍射。得出該結論后,11月30日星期一CASP發表了重大聲明:50年前的蛋白質折疊問題已經解決。

    諾貝爾獎獲得者、英國最負盛名的科學機構皇家學會(The Royal Society)現任主席文基?拉馬克里希南表示,AlphaFold 2在蛋白質折疊方面“取得了驚人的進步”。有AlphaFold 2相助,X射線晶體衍射和電子顯微鏡之類既昂貴又耗時的實證方法可能都會變成過去式。

    蛋白質結構專家、曾任歐洲分子生物學實驗室歐洲生物信息學研究所(European Molecular Biology Laboratory’s European Bioinformatics Institute)主任的珍妮特?桑頓表示,DeepMind的突破可以幫助科學家繪制出整個人類“蛋白質組”,即人體內所有蛋白質。目前人體蛋白質中只有四分之一被用作藥物靶點,如果能夠掌握其余蛋白質結構,就可以為研發新療法創造巨大的機會。她還表示,人工智能軟件還能夠推動蛋白質工程發展,從而推動可持續發展,幫科學家創造新作物品種,提升每英畝種植土地出產的營養價值,還可能研究出可以消化塑料的酶。

    不過,當前的問題仍然是DeepMind如何應用AlphaFold 2。哈薩比斯表示,公司將努力確保軟件“最大程度發揮積極的社會影響”,他也承認公司尚未決定如何實現,只說明年某個時候將宣布。哈薩比斯還告訴《財富》雜志,DeepMind正在考慮如何圍繞系統開發商業產品或建立合作伙伴關系?!跋到y對藥物研發以及制藥巨頭作用都非常大?!辈贿^他表示,商業產品的具體形式也尚未決定。

    對于DeepMind來說,如果嘗試商業化就意味著踏上新征程,而此前出售給Alphabet后公司還從來沒有擔心過收入。公司簡單成立了名叫DeepMind Health的部門,正在與英國國家醫療服務體系(U.K.’s National Health Service)合作開發應用程序,該應用程序能夠識別出存在患急性腎損傷風險的醫院患者。但新聞報道稱DeepMind的醫院合作伙伴違反英國的數據保護法向其提供數百萬患者的醫療記錄后,合作陷入了爭論。2019年,DeepMind Health正式并入新的谷歌健康部門。當時DeepMind表示,剝離健康業務可以專注自身的研究基礎,而不必分心在谷歌已然很擅長的領域(如數據安全和客戶支持)成立商業部門。

    當然了,即便DeepMind要推出商業產品,也不會是第一家嘗試商業化的人工智能研究公司??偛课挥谂f金山的OpenAI可能是最接近DeepMind的競爭對手,如今越發商業化。去年,OpenAI發布的第一個商業產品,企業能夠使用人工智能界面將簡短的手寫提示組成連貫的長文本。該人工智能被稱為GPT,商業價值尚未得到證實,而DeepMind的AlphaFold 2可能對制藥公司或生物技術初創企業產生根本性的影響。在反壟斷監管者調查Alphabet之際,擁有商業上可行的產品可能是很好的保險,以防將來拆分Googleplex時DeepMind失去財大氣粗的母公司無條件支持。

    有一點可以肯定,DeepMind在蛋白質折疊領域的探索并未結束。CASP競爭只是圍繞預測單個蛋白質的結構。在生物學和醫學領域,研究人員真正關心的通常是蛋白質如何相互作用。一種蛋白質是如何與另一種蛋白質或與某種特定的小分子結合?酶如何分解蛋白質?莫爾特說,預測相互作用和結合很可能成為未來CASP競爭的主要關注點。喬普表示,下一步DeepMind打算應對相關挑戰。

    而在蛋白質折疊以外的領域,AlphaFold 2的成功肯定也會發揮影響,將鼓勵其他人在重大科學問題中應用深入學習。比如發現新的亞原子粒子,探索暗物質的奧秘,掌握核聚變或創造室溫超導體??评锉硎?,在天體物理學方面,DeepMind已經發揮了積極的作用。Facebook的人工智能研究人員剛剛啟動了深度學習項目,希望尋找新的化學催化劑。蛋白質折疊是基礎科學當中第一個由人工智能解決的謎團,但肯定不會是最后一個。(財富中文網)

    譯者:馮豐

    審校:夏林

    2016年3月13日深夜,氣溫相當寒冷,兩名男子頭戴羊毛帽,身穿厚厚的外套,并肩走過韓國首爾市中心擁擠的街道。二人熱烈地交談,似乎完全忽視了周圍餃子館和燒烤店霓虹燈的誘惑。他們此行韓國肩負重任,多年的努力終于能夠看到結果。最棒的是,他們剛剛成功了。

    這次散步是為了慶祝。他們取得的成就將進一步鞏固他們在計算機史上的地位。在古老的戰略游戲圍棋領域里,他們開發的人工智能軟件已經充分掌握了個中奧秘,而且輕松擊敗了全球頂尖選手李世石。如今,兩人開始討論下一個目標,身后跟蹤的紀錄片攝制組捕捉到了當時的談話。

    “告訴你,我們可以解決蛋白質折疊問題?!钡旅姿?哈薩比斯對同伴大衛?西爾弗說?!澳遣攀谴蟪删?。我相信現在能夠去做了。以前我只是想過,現在肯定可以做成?!惫_比斯是總部位于倫敦的人工智能公司DeepMind的聯合創始人及首席執行官,正是該公司開發出了AlphaGo(阿爾法狗)。西爾弗則是DeepMind的計算機科學家,負責領導AlphaGo團隊。

    四年后,DeepMind實現了當年哈薩比斯在首爾散步時的設想。公司開發出了人工智能系統,能夠根據基因序列來預測蛋白質的復雜形狀,精確到單個原子寬度??恐@項成就,DeepMind完成了需要近50年才能完成的科學探索。1972年,化學家克里斯蒂安?安芬森在諾貝爾獎獲獎演說中提出,只有DNA才可以完全決定蛋白質的最終結構。這是驚人的猜想。當時連一個基因組都未完成測序。安芬森的理論開創了計算生物學的分支,目標是用復雜的數學模擬蛋白質結構,而不是實驗。

    DeepMind在圍棋方面取得的成就確實很重要,但在圍棋和計算機科學這兩個相對偏僻的領域之外,幾乎沒有產生什么具體影響。解決蛋白質折疊問題則完全不同,對大多數人來說都有變革意義。蛋白質是生命的基本組成部分,也是大多數生物過程背后的運行機制。如果能夠預測蛋白質的結構,將徹底改變人們對疾病的理解,還可以為癌癥到老年癡呆癥等各種疾病開發全新也更具針對性的藥物。新藥上市時間有望加快,藥物研發成本減少數年時間,成本也節約數億美元,還可能會拯救很多生命。

    DeepMind首創的新方法在抗擊SARS-CoV-2(也就是新冠病毒)的斗爭中已經取得成果。以下是以游戲知名的公司如何揭開生物學最大秘密的故事。

    形狀莫測的積木

    “蛋白質是細胞的主要機器?!奔又荽髮W伯克利分校的生物工程教授伊恩?霍姆斯表示。蛋白質的結構和形狀對其工作方式至關重要,構成蛋白質分子晶格的小“口袋”是發生各種化學反應的地方。如果能夠找到某種化學物質與其中一個口袋結合,這種物質就可以作為藥物阻止或加速生物過程。生物工程師還能夠創造出自然界中從未出現的全新蛋白質,而且具有獨特的療效?!叭绻覀兛梢岳玫鞍踪|的力量,合理地設計用途,就能夠制造出神奇的自我組裝機器,發揮一些作用?!被裟匪拐f。

    但為了確保蛋白質達到想要的效果,把握其形狀很重要。

    蛋白質由氨基酸鏈組成,常被比作細繩上的珠子。至于珠子按照什么順序穿起來,信息都存儲在DNA里。但是,根據簡單的基因指令很難預測完整的鏈條會形成多復雜的物理形狀。氨基酸鏈根據分子間吸引和排斥的電化學規則折疊成某種結構。形狀常常類似繩索和絲帶纏繞而成的抽象雕塑:褶皺的帶狀物加上莫比烏斯帶,就像卷曲環狀的螺旋。20世紀60年代,物理學家和分子生物學家塞勒斯?列文塔爾發現,一種蛋白質的形狀有太多可能性。如果想通過隨機嘗試組合找出蛋白質的準確結構,花的時間比已知宇宙的年齡還長。而且,幾毫秒內蛋白質就會完成折疊。該觀察被稱為列文塔爾悖論。

    到目前為止,只有通過所謂X射線晶體衍射才可以接近準確了解蛋白質的結構。顧名思義,首先需要將含有數百萬蛋白質的溶液轉化為晶體,本身就是很復雜的化學過程。然后,X射線發射到晶體上,科學家從獲得的衍射圖逆向工作,從而建立蛋白質圖像。而且,還不是隨便什么X射線都可以。要想獲得很多蛋白質的結構,要由圓形的,大小堪比體育場的同步加速器發射X射線。

    過程既昂貴又耗時。根據多倫多大學(University of Toronto)的研究人員估計,用X射線晶體衍射法測定單個蛋白質的結構需要約12個月,花費約12萬美元。已知的蛋白質超過2億種,每年大約能夠發現3000萬種,但其中只有不到20萬種蛋白質通過X射線晶體衍射或其他實驗方法繪制出了結構圖?!叭祟惖臒o知程度正在迅速增長?!庇嬎阄锢韺W家約翰?喬普說,現在他擔任DeepMind的高級研究員,負責領導蛋白質折疊團隊。

    過去50年里,自從克里斯蒂安?安芬森發表著名演講以來,科學家們一直努力使用高性能計算機上運行的復雜數學模型加速分析蛋白質結構?!盎旧暇褪菄L試在計算機里創建蛋白質的數字雙胞胎,然后嘗試操作?!瘪R里蘭大學的細胞生物學和分子遺傳學教授約翰?穆爾特說,他也是用數學算法通過DNA序列預測蛋白質結構的先驅。問題是,預測出的折疊模式經常有誤,與科學家通過X射線晶體衍射發現的結構并不一致。事實上大約10年前,很少有模型預測大蛋白質形狀時準確率可以超過三分之一。

    蛋白質折疊模擬要占用龐大的算力。2000年,研究人員創建了名叫Fold@home的“公民科學”項目,人們能夠捐出個人電腦和游戲機的閑置處理能力運行蛋白質折疊模擬。所有設備通過互聯網連接在一起,從而打造全世界最強大的虛擬超級計算機之一。大家都希望幫研究人員擺脫列文塔爾悖論,通過隨機實驗和試錯準確判斷蛋白質的結構。目前該項目仍然在進行中,已經為超過225篇論文提供了數據,研究內容是與多種疾病相關的蛋白質。

    盡管擁有強大的處理能力,Fold@home仍然深陷列文塔爾悖論,因為算法試圖搜索所有可能的排列,從而找到蛋白質結構。破解蛋白質折疊的關鍵在于跳過艱苦搜索的過程,發現蛋白質DNA序列與結構聯系的神秘模式,從而讓計算機踏上全新捷徑,直接從遺傳學領域轉到準確繪制形狀。

    嚴肅的游戲

    德米斯?哈薩比斯對蛋白質折疊的興趣始于一場游戲,他對很多事都是這樣。哈薩比斯曾經是國際象棋天才,13歲時已經成為大師,一度在同年齡里排名世界第二。他對象棋的熱愛后來轉向對兩件事感興趣:一是游戲設計,二是研究自身意識的內在機制。他高中時開始為電子游戲公司工作,在劍橋大學(University of Cambridge)學習計算機科學后,1998年創立了電腦游戲初創公司Elixir Studios。

    盡管曾經研發出兩款獲獎游戲,最終Elixir還是賣掉知識產權并關閉公司,哈薩比斯從倫敦大學學院(University College London)獲得了認知神經科學博士學位。彼時他已經開始踏上漫漫征途,后來2010年聯合創立了DeepMind。他開始研發通用人工智能軟件,不僅可以學習執行很多任務,有些甚至比人類完成得更好。哈薩比斯曾經說過,DeepMind的遠大目標是“解決智能問題,然后解決所有其他問題?!惫_比斯也曾經暗示,蛋白質折疊可能就是“其他問題”里的第一批。

    2009年,哈薩比斯在麻省理工學院(Massachusetts Institute of Technology)攻讀博士后時,聽說了一款名為Foldit的在線游戲。Foldit是由華盛頓大學(University of Washington)的研究人員設計,跟Fold@home類似,也是有關蛋白質折疊的“公民科學”項目。但Foldit并不是整合閑置的微芯片,而是利用閑置的大腦。

    Foldit是類似益智游戲的游戲,并不掌握生物學領域知識的人類玩家比賽折疊蛋白質,如果能夠得到合理的形狀就可以獲得積分。然后,研究人員分析得分最高的設計,看是否有助于破解蛋白質結構問題。游戲已經吸引成千上萬玩家,并且一些記錄案例中得到的蛋白質結構比研究蛋白質折疊的計算機算法更準確?!皬倪@個角度來看,我覺得游戲很有趣,想著能不能利用游戲的上癮性和游戲的樂趣,不僅讓人們玩得開心,也做一些對科學有用的事情?!惫_比斯說。

    Foldit能夠抓住哈薩比斯的想象力還有另一個原因。其實游戲是一種強化學習行為,特別適合訓練人工智能。軟件可以通過試驗和試錯從經驗中學習,從而更好地完成任務。在游戲里軟件能夠無休止地試驗,反復地玩,逐步改進,不對現實世界造成傷害的情況下提升技能水平,直到超過人類。游戲也有現成的方法判斷某個特定的動作或某組動作是否有效,即積分和勝利。種種指標可以提供非常明確的標準衡量表現,在現實世界很多問題里則無法如此處理?,F實世界遇到問題時,最有效的方法可能比較模糊,“獲勝”的概念也可能不適用。

    DeepMind的基礎主要是將強化學習與稱為深度學習的人工智能相結合。深度學習是基于神經網絡的人工智能,所謂神經網絡是大致基于人腦工作原理的軟件。這種情況下,軟件沒有實際的神經細胞網絡,而是一堆虛擬神經元分層排列,初始輸入層接收數據,按照權重分配后傳遞到中間層,中間層依次執行相同操作,最終傳遞到輸出層,輸出層匯總各項加權值并算出結果。網絡能夠調整各項權重,直到產生理想的結果,例如準確識別貓的照片或國際象棋獲勝。之所以被稱為“深度學習”,并不是因為產生的結果一定深刻,當然也有可能深刻,但主要原因是網絡由許多層構成,所以可以說具有深度。

    DeepMind最初成功是用“深度強化學習”創建軟件,自學玩經典的雅達利電腦游戲,如《乒乓球》(Pong)、《突圍》(Breakout)和《太空入侵者》(Space Invaders)等,而且水平超過人類。正是這一成就讓DeepMind受到谷歌(Google)等科技巨頭的關注,據報道,2014年谷歌以4億英鎊(當時超過6億美元)收購了DeepMind。之后公司主攻圍棋并開發了AlphaGo系統,2016年擊敗了李世石。DeepMind接著開發了名叫AlphaZero的更通用系統版本,幾乎能夠學會所有兩玩家回合制游戲,在這種游戲中,玩家都可以獲得充分信息(沒有機會隱藏信息,例如牌面朝下放置或隱藏位置)。去年,公司開發的系統還在高度復雜的即時戰略游戲《星際爭霸2》(Starcraft 2)中擊敗了頂尖的人類職業電競玩家。

    但哈薩比斯表示,一直認為公司在游戲方面的探索是完善人工智能系統的方式,之后能夠應用于現實世界挑戰,尤其是科學領域?!氨荣愔皇怯柧殘?,但訓練到底為了什么?最終是為了創造新知識?!彼f。

    DeepMind并非具有產品和客戶的傳統業務,本質上是推動人工智能前沿的研究實驗室。公司的很多開發方法都已經公開,供所有人使用或借鑒。不過某些方面的進步對姊妹公司谷歌也頗有幫助。

    DeepMind團隊由工程師和科學家組成,幫助谷歌將尖端的人工智能技術融入產品。DeepMind的技術已經滲透各處,從谷歌地圖(Google Maps)到數字助理,再到協助管理安卓手機電池電量的系統。谷歌為此向DeepMind支付費用,母公司Alphabet繼續承擔DeepMind帶來的額外虧損。虧損規模并不小,2018年,公司虧損4.7億英鎊(當時約合5.1億美元),這也是通過英國的商業注冊機構公司登記局(Companies House)可以查到的最新一年公開記錄。

    不過如今員工超過1000人的DeepMind,還有一整個部門只負責人工智能的科學應用。該部門的負責人為39歲的印度人普什米?科里,他加入DeepMind之前曾經在微軟從事人工智能研究。他表示,DeepMind的目標是解決“根節點”問題,這是數據科學家的慣用語,意思是希望解決能夠解鎖很多科學路徑的基礎問題。蛋白質折疊就是根節點之一,科里說。

    “蛋白質折疊的奧運會”

    1994年,當很多科學家剛開始使用復雜的計算機算法預測蛋白質折疊方式時,馬里蘭大學的生物學家墨爾特決定開辦競賽,用公正的方法評估哪種算法最好。他把比賽稱為蛋白質結構預測關鍵評估(簡稱為CASP),之后每兩年舉辦一次。

    賽事具體如下,美國國立衛生研究院資助的蛋白質結構預測中心主辦CASP,并說服從事X射線晶體衍射和其他實證研究的研究人員提供尚未公布的蛋白質結構,要求在CASP競賽結束之前不公開相關結構。然后CASP將蛋白質DNA序列發給參賽者,參賽者用算法預測蛋白質結構。CASP判斷預測與X射線晶體學家和實驗學家發現的實際結構接近程度,然后根據算法對各種蛋白質預測的平均得分排名?!拔曳Q之為蛋白質折疊界的奧運會?!惫_比斯說。2016年AlphaGo擊敗李世石后不久,DeepMind就打算贏得金牌。

    DeepMind組建了小規模精干的團隊,由六名機器學習研究人員和工程師組成?!白尅ú拧胧质俏覀兊睦砟??!惫_比斯說。公司里并不缺乏人才?!扒拔锢韺W家、前生物學家,大家都四處閑逛?!惫_比斯有點啼笑皆非?!八麄冇肋h不知道之前的專業知識什么時候可以突然發揮作用?!弊詈髨F隊成員增加到20人左右。

    不過,DeepMind還是認為團隊里至少要有一位真正的蛋白質折疊專家,后來選中了約翰?喬普。35歲的喬普像個大男孩,瘦得皮包骨,一頭蓬亂斜梳的棕色頭發,有點像20世紀90年代末高中車庫樂隊的低音吉他手。他在劍橋大學獲得理論凝聚態物理碩士學位,之后在紐約由對沖基金億萬富翁大衛?肖創立的獨立研究實驗室D.E.Shaw Research工作。實驗室專門研究計算生物學,包括蛋白質模擬。后來喬普在芝加哥大學獲得了計算生物物理學博士學位,導師為卡爾?弗里德和托賓?索斯尼克,兩位科學家皆因推動蛋白質折疊模型進步出名?!拔以浡犝fDeepMind對解決蛋白質結構有興趣?!彼f。于是他申請并順利加入。

    哈薩比斯和DeepMind團隊的第一直覺是,蛋白質折疊能夠用與圍棋完全相同的方式解決,即深度強化學習。事實證明存在問題。首先,蛋白質折疊結構的可能性比圍棋的步數還要多。更重要的是,DeepMind讓工智能系統AlphaGo與自己對弈就可以掌握圍棋的玩法?!八钥杀刃圆⒉桓?,因為蛋白質折疊不是雙人游戲?!惫_比斯說,“有點違背自然?!?/p>

    DeepMind很快發現,如果使用所謂監督式深度學習的人工智能培訓方法,就能夠更簡便地取得進步。這是大多數商業應用里使用的人工智能,神經網絡通過一組既定數據輸入和相應輸出,可以學習如何將給定的輸入與給定輸出相匹配。具體到蛋白質結構,DeepMind已經掌握約170000個蛋白質結構,能夠作為訓練數據。蛋白質數據庫(PDB)是已知三維蛋白質形狀及遺傳序列的公共存儲庫,可以公開查詢相關結構。

    一些生物學家已經使用監督式深度學習預測蛋白質如何折疊。但此類人工智能系統表現最佳的正確率也只有50%,對生物學家或醫學研究人員沒有什么幫助,尤其是對結構未知的蛋白質,因為無法確定某次特定預測是否正確。

    有種技術很有希望,其理念是基于蛋白質的進化史劃分為不同的家族。各種家族里可能在一個DNA序列中找到相距遙遠但似乎會同時突變的氨基酸對。此類所謂“共同進化”的現象很有幫助,因為共同進化的蛋白質很可能在蛋白質折疊結構中有聯系。位于芝加哥的豐田技術研究所(Toyota Technological Institute)的科學家徐金波(音譯)率先利用深入學習共同進化數據預測氨基酸聯系。這種方法有點像是在連接點游戲里尋找點??茖W家仍然要用其他軟件找出點之間的線,過程中經常出錯。有時候連點都找不準。

    在2018年的CASP競賽中,DeepMind應用了共同進化和預測聯系的基本思想,但增加了兩個重要的轉折點。首先,系統沒有試圖確定兩個氨基酸是否有聯系,也就是二進制輸出(即兩個氨基酸可能有聯系,也可能沒有聯系),而是決定讓算法預測蛋白質里所有氨基酸對之間的距離。

    在多數分子生物學家看來,這種方法似乎違反直覺,不過值得稱贊的是,徐金波也獨立提出了類似方法。畢竟,聯系才是最重要的。對于DeepMind的深度學習專家來說,很明顯距離是讓神經網絡發揮作用更好的指標,科里表示?!斑@只是深度學習的基礎部分,如果與決策相關存在不確定性,最好是讓神經網絡整合不確定性,并決定如何應對?!彼f。與聯系不一樣,距離包含了神經網絡可調整和使用的豐富信息。

    DeepMind另一項讓人意外之處是引入第二個神經網絡,用于預測氨基酸對之間的角度。有了距離和角度兩個因素,DeepMind的算法就能夠算出蛋白質結構的大致輪廓。然后,系統使用另一種非人工智能算法改進結構。DeepMind將相關組件整合到名為AlphaFold的系統中,橫掃了2018年CASP(又稱為第13屆CASP,因為是兩年一度比賽舉辦第13次。)比賽里結構最復雜的43種蛋白質中,AlphaFold在25種蛋白質中得分最高。第二名僅在三種蛋白質里得到高分。研究結果震驚了全行業。如果說之前還有人懷疑深度學習究竟是不是解決蛋白質折疊問題最有希望的方法,AlphaFold讓所有人再無疑問。

    回到白板

    盡管如此,DeepMind還遠沒有達到哈薩比斯的目標,即完全解決蛋白質折疊問題。AlphaFold準確率只有一半,第13屆CASP的104個蛋白質中,準確度可以達到X射線晶體衍射水平的只有三個?!拔覀儾恢幌朐贑ASP競賽中奪魁,而是想真正解決問題。我們想打造對生物學家很重要的系統?!眴唐照f。

    2018年CASP的結果公布后不久,DeepMind就開始加倍努力。喬普負責擴大的團隊。團隊并未簡單地在AlphaFold基礎上改進,而是返回原點,集思廣益尋找完全不同的想法,他們希望新創意能夠幫軟件將精確度提升到更接近X射線晶體衍射級別。

    喬普表示,接下來是整個項目中最可怕也最令人沮喪的時期之一,因為什么辦法都沒有?!拔覀兓巳齻€月,結果都達不到第13屆CASP的水平,開始真正感覺到恐慌?!彼f。不過當時研究人員的嘗試出現了一些改進,沒到6個月系統已經比最初的AlphaFold有了明顯改進。之后兩年里一直延續該模式,喬普說。先是三個月一無所獲,接下來三個月快速發展,接著又是平臺期。

    哈薩比斯說,DeepMind以前的項目也出現過類似模式,包括圍棋項目,還有復雜的即時戰略游戲《星際爭霸2》項目。他說,公司克服問題的管理策略就是交替采取兩種不同的工作方式。第一種哈薩比斯稱之為“攻擊模式”,盡可能推動團隊,追求當前系統可以達到的極致表現。然后,全力以赴努力的效果似乎耗盡時,他就開始轉向所謂的“創新模式”。期間哈薩比斯不再對團隊施加壓力,容忍甚至期待出現暫時性的后退,從而為研究人員和工程師提供修補新想法和嘗試新手段的空間。他說:“要鼓勵人們提出盡可能多的瘋狂想法,還要頭腦風暴?!痹撃J酵ǔD軌蛲苿有阅艹霈F新飛躍,讓團隊切換回攻擊模式。

    生日大禮

    2019年11月21日,DeepMind蛋白質折疊團隊的研究員凱薩倫?圖雅蘇那科年滿30歲。這一天也會因為另一個原因值得紀念。圖雅蘇那科擁有牛津大學(University of Oxford)計算生物學博士學位,在團隊里負責為蛋白質折疊人工智能開發新測試集,新款人工智能叫AlphaFold 2,是DeepMind為2020年的CASP競賽新開發的系統。那天早上她打開辦公電腦時,收到系統對一批大約50個蛋白質序列預測的評估,所有序列均為最近才添加到蛋白質數據庫中。她愣了一下,然后大吃一驚。AlphaFold 2確實一直在改進,但對該組蛋白質的預測結果驚人地準確。系統對好幾個蛋白質結構結構預測誤差在1.5埃以內,埃的距離單位相當于十分之一納米,或大約一個原子的寬度。

    自稱“團隊悲觀主義者”的圖雅蘇那科說,第一反應并不是高興而是有點想吐?!拔耶敃r很害怕?!彼f。結果實在太好,她以為是自己犯了錯,可能準備測試集時無意中把人工智能在訓練數據里見過的幾個蛋白質加了進來。如此一來AlphaFold 2基本上就可以作弊,輕易預測出準確的結構。圖雅蘇那科回憶說,當時坐在DeepMind自助餐廳俯瞰倫敦的圣潘克拉斯車站(St. Pancras Station),一杯接一杯地喝茶努力平復心情。隨后,她和其他團隊成員花了一整天,直到深夜才下班,之后幾天也是如此,他們坐在工作站旁埋頭梳理AlphaFold 2的訓練數據,希望找出錯誤所在。

    然而一個錯誤也沒有。事實是,新系統在預測表現方面實現了巨大飛躍。AlphaFold 2與之前版本完全不同。人工智能不再只是各成分組合,一個用來預測氨基酸之間的距離,另一個預測角度,然后用第三個軟件聯系起來?,F在的人工智能用單一的神經網絡直接從DNA序列進行推理。雖然系統仍然接受進化信息,從而確定研究的蛋白質是否與以前見過的蛋白質有共同的祖先,并仔細檢查目標蛋白質的DNA序列與其他已知序列之間的一致性,但不再需要哪些氨基酸對共同進化的明確數據?!拔覀儾⑽刺峁└嘈畔?,反而減少了信息?!眴唐照f。系統可以自由地得出見解,即祖先何時可能決定蛋白質的部分形狀,以及何時可能徹底偏離。換句話說,系統根據經驗培養出直覺,就像老練的人類科學家一樣。

    新系統的核心是“注意力”機制,顧名思義,注意力是讓深度學習系統專注于某組輸入,并對相關輸入加大權重。舉例來說,在識別貓的系統里,系統可能學會注意耳朵的形狀,也會學習在鼻子附近尋找胡須。喬普比較了AlphaFold 2的功能與玩拼圖游戲,過程中“能夠將某些部分拼湊在一起而且非常確定,得到不同的本地解決方案,然后想辦法將相關問題連接起來?!眴唐照f,神經網絡的中層已經學會根據對DNA序列的分析推理幾何和空間排列,以及氨基酸對如何連接。

    DeepMind曾經在128個“張量處理核心”上訓練AlphaFold 2,張量處理核心是在16塊專門用于深度學習的計算機芯片上創建的數字運算大腦,芯片由谷歌設計并在數據中心使用,公司稱連續運行了數周。(128個專用的人工智能核心大約相當于100到200塊強大的圖形處理芯片,可以在Xbox或PlayStation上呈現極其炫目的動畫效果。)公司表示,經過訓練的系統提取DNA序列后“幾天內”就能夠完成整個結構預測。

    AlphaFold 2與前一代相比有個優勢,就是提供可信程度,即系統對結構里每種氨基酸的預測都有信心分數。如果說AlphaFold 2可以切實幫到生物學家和醫學研究人員,這項指標至關重要,因為研究者需要清楚何時能夠合理依賴模型,以及何時需要更加謹慎。

    盡管測試結果驚人,DeepMind仍然不能確定AlphaFold 2的預測效果。新冠病毒來襲時,公司才得到重要的線索。今年3月,AlphaFold 2可以預測出六種與SARS-CoV-2(引發疫情的病毒)相關但未被研究的蛋白質結構,后來科學家使用所謂低溫電子顯微鏡的經驗方法證實了其中一種。由此能夠充分看出AlphaFold 2對現實世界的影響力。

    驚人的結果

    CASP比賽在5月到8月之間舉行。蛋白質結構預測中心發布多批目標蛋白質,之后參賽方提交結構預測進行評估。今年比賽排名于11月30日公布。

    每次預測均可以得到“全球距離測試總分”,簡稱GDT的指標評分,該指標實際上看預測結果與通過實證方法(如X射線晶體衍射或電子顯微鏡)得到的結構接近程度,單位為埃。CASP的主席穆爾特表示,滿分是100分,如果得分能夠達到90分或以上,說明與實證方法相當。根據CASP組織者判斷的結構難度,蛋白質也會劃分不同的組。

    穆爾特看到AlphaFold 2的結果時簡直不敢相信。他就像幾個月前的圖雅蘇那科一樣,剛開始的想法是出錯了。也許比賽中一些蛋白質序列以前發表過?又或者DeepMind也許設法獲得了未發布數據的緩存?

    為了核實,他請位于德國圖賓的根馬克斯?普朗克發展生物學研究所(Max Planck Institute for Developmental Biology)的蛋白質進化系主任安德烈?盧帕斯幫忙驗證。盧帕斯讓AlphaFold 2預測一個自己確信沒有見過的結構,因為盧帕斯利用X射線結晶衍射從未成功觀測到該蛋白質的關鍵部分。近十年來,盧帕斯一直因為該部分缺失而傷腦筋,但就是觀測不到準確的形狀。盧帕斯說,利用AlphaFold的預測后,他重新查看X射線數據?!皼]到半小時就得出了正確結構?!彼f,“太令人吃驚了!”

    2018年DeepMind在CASP中獲得成功以來,諸多學術研究人員紛紛涌向深度學習技術。結果,該領域其他方面的表現都有所提高。在中等難度目標方面,其他競爭對手的平均最佳預測GDT得分為75,比兩年前提高了10分。不過還是完全追不上AlphaFold 2,因為該系統預測蛋白質結構平均得分高達92,就算面對最復雜的蛋白質平均得分也有87。穆爾特表示AlphaFold 2的預測“與實證方法不相上下”,比如X射線晶體衍射。得出該結論后,11月30日星期一CASP發表了重大聲明:50年前的蛋白質折疊問題已經解決。

    諾貝爾獎獲得者、英國最負盛名的科學機構皇家學會(The Royal Society)現任主席文基?拉馬克里希南表示,AlphaFold 2在蛋白質折疊方面“取得了驚人的進步”。有AlphaFold 2相助,X射線晶體衍射和電子顯微鏡之類既昂貴又耗時的實證方法可能都會變成過去式。

    蛋白質結構專家、曾任歐洲分子生物學實驗室歐洲生物信息學研究所(European Molecular Biology Laboratory’s European Bioinformatics Institute)主任的珍妮特?桑頓表示,DeepMind的突破可以幫助科學家繪制出整個人類“蛋白質組”,即人體內所有蛋白質。目前人體蛋白質中只有四分之一被用作藥物靶點,如果能夠掌握其余蛋白質結構,就可以為研發新療法創造巨大的機會。她還表示,人工智能軟件還能夠推動蛋白質工程發展,從而推動可持續發展,幫科學家創造新作物品種,提升每英畝種植土地出產的營養價值,還可能研究出可以消化塑料的酶。

    不過,當前的問題仍然是DeepMind如何應用AlphaFold 2。哈薩比斯表示,公司將努力確保軟件“最大程度發揮積極的社會影響”,他也承認公司尚未決定如何實現,只說明年某個時候將宣布。哈薩比斯還告訴《財富》雜志,DeepMind正在考慮如何圍繞系統開發商業產品或建立合作伙伴關系?!跋到y對藥物研發以及制藥巨頭作用都非常大?!辈贿^他表示,商業產品的具體形式也尚未決定。

    對于DeepMind來說,如果嘗試商業化就意味著踏上新征程,而此前出售給Alphabet后公司還從來沒有擔心過收入。公司簡單成立了名叫DeepMind Health的部門,正在與英國國家醫療服務體系(U.K.’s National Health Service)合作開發應用程序,該應用程序能夠識別出存在患急性腎損傷風險的醫院患者。但新聞報道稱DeepMind的醫院合作伙伴違反英國的數據保護法向其提供數百萬患者的醫療記錄后,合作陷入了爭論。2019年,DeepMind Health正式并入新的谷歌健康部門。當時DeepMind表示,剝離健康業務可以專注自身的研究基礎,而不必分心在谷歌已然很擅長的領域(如數據安全和客戶支持)成立商業部門。

    當然了,即便DeepMind要推出商業產品,也不會是第一家嘗試商業化的人工智能研究公司??偛课挥谂f金山的OpenAI可能是最接近DeepMind的競爭對手,如今越發商業化。去年,OpenAI發布的第一個商業產品,企業能夠使用人工智能界面將簡短的手寫提示組成連貫的長文本。該人工智能被稱為GPT,商業價值尚未得到證實,而DeepMind的AlphaFold 2可能對制藥公司或生物技術初創企業產生根本性的影響。在反壟斷監管者調查Alphabet之際,擁有商業上可行的產品可能是很好的保險,以防將來拆分Googleplex時DeepMind失去財大氣粗的母公司無條件支持。

    有一點可以肯定,DeepMind在蛋白質折疊領域的探索并未結束。CASP競爭只是圍繞預測單個蛋白質的結構。在生物學和醫學領域,研究人員真正關心的通常是蛋白質如何相互作用。一種蛋白質是如何與另一種蛋白質或與某種特定的小分子結合?酶如何分解蛋白質?莫爾特說,預測相互作用和結合很可能成為未來CASP競爭的主要關注點。喬普表示,下一步DeepMind打算應對相關挑戰。

    而在蛋白質折疊以外的領域,AlphaFold 2的成功肯定也會發揮影響,將鼓勵其他人在重大科學問題中應用深入學習。比如發現新的亞原子粒子,探索暗物質的奧秘,掌握核聚變或創造室溫超導體??评锉硎?,在天體物理學方面,DeepMind已經發揮了積極的作用。Facebook的人工智能研究人員剛剛啟動了深度學習項目,希望尋找新的化學催化劑。蛋白質折疊是基礎科學當中第一個由人工智能解決的謎團,但肯定不會是最后一個。(財富中文網)

    譯者:馮豐

    審校:夏林

    It is March 13, 2016. Two men, dressed in winter coats and woolen hats to defend against the frigid night air, walk side by side through the crowded streets of downtown Seoul. Locked in animated conversation, they seem oblivious to the pulsating neon enticements of the surrounding dumpling houses and barbecue joints. They are visitors, having come to South Korea on a mission, the culmination of years of effort—and they have just succeeded.

    This is a celebratory stroll. What they have achieved will cement their places in the annals of computer science: They have built a piece of artificial intelligence software able to play the ancient strategy game Go so expertly that it handily defeated the world’s top player, Lee Sedol. Now the two men are discussing their next goal, their conversation captured by a documentary film crew shadowing them.

    “I’m telling you, we can solve protein folding,” Demis Hassabis says to his walking companion, David Silver. “That’s like, I mean, it’s just huge. I am sure we can do that now. I thought we could do that before, but now we definitely can do it.” Hassabis is the cofounder and chief executive officer of DeepMind, the London-based A.I. company that built AlphaGo. Silver is the DeepMind computer scientist who led the AlphaGo team.

    Four years later, DeepMind has just accomplished what Hassabis broached in that nocturnal amble: It has created an A.I. system that can predict the complex shapes of proteins down to an atom’s-width accuracy from the genetic sequences that encode them. With this achievement, DeepMind has completed an almost 50-year-old scientific quest. In 1972, in his Nobel Prize acceptance speech, chemist Christian Anfinsen postulated that DNA alone should fully determine the final structure a protein takes. It was a remarkable conjecture. At the time, not a single genome had been sequenced yet. But Anfinsen’s theory launched an entire subfield of computational biology with the goal of using complex mathematics, instead of empirical experiments, to model proteins.

    DeepMind’s achievement with Go was important—but it had little concrete impact outside the relatively cliquish worlds of Go and computer science. Solving protein folding is different: It could prove transformative for much of humanity. Proteins are the basic building blocks of life and the mechanism behind most biological processes. Being able to predict their structure could revolutionize our understanding of disease and lead to new, more targeted pharmaceuticals for disorders from cancer to Alzheimer’s disease. It will likely accelerate the time it takes to bring new medicines to market, potentially shaving years and hundreds of millions of dollars in costs from drug development, and potentially saving lives as a result.

    The new method pioneered by DeepMind is already yielding results in the fight against SARS-CoV-2, the virus that causes COVID-19. What follows is the story of how a company best known for playing games came to unlock one of biology’s greatest secrets.

    Building blocks with elusive shapes

    “Proteins are the main machines of the cell,” Ian Holmes, a professor of bioengineering at the University of California at Berkeley, says. “And the structure and shape of them is crucial to how they operate.” Small “pockets” within the lattice of molecules that make up the protein are where various chemical reactions take place. If you can find a chemical that will bind to one of these pockets, then that substance can be used as a drug—to either disable or accelerate a biological process. Bioengineers can also create entirely new proteins never before seen in nature with unique therapeutic properties. “If we could tap into the power of proteins and rationally engineer them to any purpose, then we could build these remarkable self-assembling machines that could do things for us,” Holmes says.

    But to be sure the protein will do what you want, it’s important to know its shape.

    Proteins consist of chains of amino acids, often compared to beads on a string. The recipe for which beads to string in what order is encoded in DNA. But the complex physical shape the completed chain will take is extremely difficult to predict from those simple genetic instructions. Amino acid chains collapse—or fold—into a structure based on electrochemical rules of attraction and repulsion between molecules. The resulting shapes frequently resemble abstract sculptures formed from tangles of cord and ribbon: pleated banderoles joined to M?bius strip–like curlicues and looping helixes. In the 1960s, Cyrus Levinthal, a physicist and molecular biologist, determined that there were so many plausible shapes a protein might assume that it would take longer than the known age of the universe to arrive at the correct structure by randomly trying combinations—and yet, the protein folds itself in milliseconds. This observation has become known as Levinthal’s Paradox.

    Until now, the only way to know a protein’s structure with near certainty was through a method known as X-ray crystallography. As the name implies, this involves turning solutions of millions of proteins into crystals, a chemical process that is itself tricky. X-rays are then fired at these crystals, allowing a scientist to work backward from the diffraction patterns they make to build up a picture of the protein itself. Oh, and not just any X-rays: For many proteins, the X-rays need to be produced by a massive, stadium-size circular particle accelerator called a synchrotron.

    The process is expensive and time-consuming: It takes about 12 months and approximately $120,000 to determine a single protein’s structure with X-ray crystallography, according to one estimate from researchers at the University of Toronto. There are over 200 million known proteins, with about 30 million more being discovered every year, and yet fewer than 200,000 of these have had their structures mapped with X-ray crystallography or other experimental methods. “Our level of ignorance is growing rapidly,” says John Jumper, a computational physicist who is now a senior researcher at DeepMind and leads its protein-folding team.

    Over the past 50 years, ever since Christian Anfinsen’s famous speech, scientists have tried speed up the analysis of protein structure by using complex mathematical models run on high-powered computers. “What you do is essentially try to create a digital twin of the protein in your computer, and then try to manipulate it,” says John Moult, a professor of cell biology and molecular genetics at the University of Maryland and a pioneer in using mathematical algorithms to predict protein structures from their DNA sequences. The problem is, these predicted folding patterns were frequently wrong, failing to match the structures scientists found through X-ray crystallography. In fact, until about 10 years ago, few models were able to accurately predict more than about a third of a large protein’s shape.

    Some protein-folding simulations also take up tremendous amounts of computing power. In the year 2000, researchers created a “citizens science” project called Fold@home in which people could donate the idle processing capacity of their personal computers and game consoles to run a protein-folding simulation. All those devices, chained together through the Internet, created one of the world’s most powerful virtual supercomputers. The hope was that this would allow researchers to escape Levinthal’s Paradox—to speed up the time it would take to hit upon the accurate protein structures through random trial and error. The project, which is still running, has provided data for more than 225 scientific papers on proteins implicated in a number of diseases.

    But despite having access to so much processing power, Fold@home is still mired in Levinthal's Paradox: It is trying to find a protein structure by searching through all possible permutations. The holy grail of protein folding is to skip this laborious search and to instead discover elusive patterns that link a protein’s DNA sequence to its structure—allowing a computer to take a radical shortcut, leaping directly from genetics to the correct shape.

    Games with a serious purpose

    Demis Hassabis’s interest in protein folding began, as many of Hassabis’s passions do, with a game. Hassabis is a former chess prodigy, a master by the time he was 13 and at one time ranked second in the world for his age. His love of chess fed a fascination with two things: game design and the inner mechanisms of his own mind. He began working for a video games company while still in high school and, after studying computer science at the University of Cambridge, founded his own computer games startup, Elixir Studios, in 1998.

    Despite producing two award-winning games, Elixir eventually sold off its intellectual property and shut down, and Hassabis went on to get a Ph.D. in cognitive neuroscience from University College London. By then, he had already embarked on the crusade that would lead him to cofound DeepMind in 2010: the creation of artificial general intelligence—software capable of learning to perform many disparate tasks as well or better than people. DeepMind’s lofty goal, Hassabis once said, was “to solve intelligence, and then use it to solve everything else.” Hassabis already had an inkling that protein folding just might be one of those first “everything elses.”

    Hassabis was doing a postdoc at the Massachusetts Institute of Technology in 2009 when he heard about an online game called Foldit. Foldit was designed by researchers at the University of Washington and, like Fold@home, it was a “citizens science” project for protein folding. But instead of yoking together idle microchips, Foldit was designed to harness idle brains.

    Foldit is a puzzle-like game in which human players, without any knowledge of biology, compete to fold proteins, earning points for creating shapes that are plausible. Researchers then analyze the highest-scoring designs to see if they can help complete unsolved protein structures. The game has attracted tens of thousands of players and, in a number of documented cases, produced better protein structures than protein-folding computer algorithms. “I thought it was fascinating from the standpoint of, can we use the addictiveness of games and the joy of them, and in the background not only are they having fun, but they are doing something useful for science,” Hassabis says.

    But there was another reason Foldit would continue to capture Hassabis’s imagination. Games are a particularly good arena for a kind of A.I. training called reinforcement learning. This is where software learns from experience, essentially by trial and error, to get better at a task. In a computer game, software can experiment endlessly, playing over and over again, improving gradually until it reaches superhuman skill, without causing any real-world harm. Games also have ready-made and unambiguous ways to tell if a particular action or set of actions is effective: points and wins. Those metrics provide a very clear way to benchmark performance—something that doesn’t exist for many real-world problems, where the most effective move may be far more ambiguous and the entire concept of “winning” may not apply.

    DeepMind was founded largely on the promise of combining reinforcement learning with a kind of A.I. called deep learning. Deep learning is A.I. based on neural networks—a kind of software loosely based on how the human brain works. In this case, instead of networks of actual nerve cells, the software has a bunch of virtual neurons, arranged in a hierarchy where an initial input layer takes in some data, applies a weighting to it, and passes it along to the middle layers, which do the same in turn, until it is eventually passed to an output layer that sums up all the weighted signals and uses that to produce a result. The network adjusts these weights until it can produce a desired outcome—such as accurately identifying photos of cats or winning a game of chess. It’s called “deep learning” not because the insights it produces are necessarily profound—although they can be—but because the network consists of many layers and so can be said to have depth.

    DeepMind’s initial success came in using this “deep reinforcement learning” to create software that taught itself to play classic Atari computer games, such as Pong, Breakout, and Space Invaders, at superhuman levels. It was this achievement that helped get DeepMind noticed by big technology firms, including Google, which bought it for a reported £400 million (more than $600 million at the time) in 2014. It then turned its attention to Go, eventually creating the system AlphaGo, which defeated Sedol in 2016. DeepMind went on to create a more general version of that system, called AlphaZero, that could learn to play almost any two-player, turn-based game in which players have perfect information (so there is no element of chance or hidden information, such as face-down cards or hidden positions) at superhuman levels. Last year, it also built a system that could beat top human professional e-sports players at the highly complex real-time strategy game Starcraft 2.

    But Hassabis says he always saw the company’s work with games as a way to perfect A.I. methods so they could be applied to real-world challenges—especially in science. “Games are just a training ground, but a training ground for what exactly? For creating new knowledge,” he says.

    DeepMind is not a traditional business, with products and customers. Instead, it is essentially a research lab that tries to advance the frontiers of artificial intelligence. Many of the methods it develops, it publishes openly for anyone to use or build upon. But some of its advances are useful for its sister company, Google.

    DeepMind has a whole team of engineers and scientists that help Google incorporate cutting-edge A.I. into its products. DeepMind’s technology has found its way into everything from Google Maps to the company’s digital assistant to the system that helps manage battery power on Android phones. Google pays DeepMind for this help, and Alphabet, its parent company, continues to absorb the additional losses that DeepMind generates. Those are not insignificant: The company lost £470 million in 2018 (about $510 million at the time), the last year for which its annual financial statements are publicly available through the U.K. business registry Companies House.

    But DeepMind, which now employs more than 1,000 people, also has a whole other division that works only on scientific applications of A.I. It is headed by Pushmeet Kohli, a 39-year-old native of India, who worked on A.I. research for Microsoft before joining DeepMind. He says that DeepMind’s aim is to try to solve “root node” problems—data science-speak for saying it wants to take on issues that are fundamental to unlocking many different scientific avenues. Protein folding is one of these root nodes, Kohli says.

    “The Olympics of protein folding”

    In 1994, at a time when many scientists were first starting to use sophisticated computer algorithms to try to predict how proteins would fold, Moult, the University of Maryland biologist, decided to create a competition that could provide an unbiased way of assessing which of these algorithms was best. He called this competition the Critical Assessment of Protein Structure Prediction (CASP, for short), and it has been held biennially ever since.

    It works like this: The Protein Structure Prediction Center, the organization that runs CASP and which is funded through the U.S. National Institute of General Medical Sciences, persuades researchers who do X-ray crystallography and other empirical studies to provide it with protein structures that have not yet been published anywhere, asking them to refrain from making the structures public until after the CASP competition. CASP then gives the DNA sequences of these proteins to the contestants, who use their algorithms to predict the protein’s structure. CASP then judges how close the predictions are to the actual structure the X-ray crystallographers and experimentalists found. The algorithms are then ranked by their average performance across all the proteins. “I call it the Olympics of protein folding,” Hassabis says. And, in 2016, shortly after AlphaGo beat Sedol, DeepMind set out to win the gold medal.

    DeepMind established a small, crack team of a half-dozen machine learning researchers and engineers to work on the problem. “It’s part of our philosophy that we start with generalists,” Hassabis says. The company does not suffer from a lack of brain power. “Ex-physicists, ex-biologists, we just have them lying around generally,” Hassabis says with a wry smile. “They never know when their previous expertise suddenly is going to become useful.” Eventually the team grew to about 20 people.

    Still, DeepMind decided it would be helpful to have at least one true protein-folding expert onboard. It found one in John Jumper. Skinny, with a mop of asymmetrically styled brown hair, Jumper is a boyish 35 and looks a bit like the bass guitarist in a late-1990s high school garage band. He earned a master’s degree in theoretical condensed matter physics from Cambridge before going on to work at D.E. Shaw Research in New York City, an independent research lab founded by hedge fund billionaire David Shaw. The lab specializes in computational biology, including the simulation of proteins. Jumper later got his Ph.D. in computational biophysics from the University of Chicago, studying under Karl Freed and Tobin Sosnick, two scientists known for advances in protein-fold modeling. “I had heard this rumor that DeepMind was interested in protein problems,” he says. He applied and got the job.

    Hassabis’s and the DeepMind team’s first instinct was that protein folding could be solved in exactly the same way as Go—with deep reinforcement learning. But this proved problematic: For one thing, there were even more possible fold configurations than there are moves in Go. More importantly, DeepMind had mastered Go in large part by getting its A.I. system, AlphaGo, to play games against itself. “There isn’t quite the right analogy for that because protein folding is not a two-player game,” Hassabis says. “You’re sort of playing against Nature.”

    DeepMind soon established that there was a simpler way of making progress using a kind of A.I. training known as supervised deep learning. This is the sort of A.I. used in most business applications: From an established set of data inputs and corresponding outputs, a neural network learns how to match a given input to a given output. In this case, DeepMind had the protein structures—currently about 170,000 of them—that are publicly available in the Protein Data Bank (PDB), a public repository of all known three-dimensional protein shapes and their genetic sequences, to use as training data.

    Some biologists had already used supervised deep learning to predict how proteins would fold. But the best of these A.I. systems were right only about 50% of the time, which wasn’t particularly helpful to biologists or medical researchers—especially since, for a protein whose structure was unknown, they had no way of determining whether a particular prediction was correct.

    One promising technique rested on the idea that proteins can be grouped into families based on their evolutionary history. And within these families, it is possible to find pairs of amino acids that are distant from one another in a DNA sequence, yet seem to mutate at the same time. This phenomenon, which is called “coevolution,” is helpful because coevolved proteins are likely to be in contact within the protein’s folded structure. Jinbo Xu, a scientist at the Toyota Technological Institute in Chicago, pioneered using deep learning on this coevolutionary data to predict amino acid contacts. The approach is a bit like finding just the dots in a connect-the-dots game. Scientists still had to use other software to try to figure out the lines between those dots—and often they got this wrong. Sometimes they didn’t even get the dots right.

    For the 2018 CASP competition, DeepMind took these basic ideas about coevolution and contact prediction but added two important twists. First, rather than trying to determine if two amino acids were in contact, a binary output (either the pair is in contact or isn’t), it decided to ask the algorithm to predict the distance between all the amino acid pairs in the protein.

    To most molecular biologists, such an approach seemed counterintuitive—although Xu, to his credit, had independently proposed a similar method. After all, it was contact that mattered most. But to DeepMind’s deep learning experts it was immediately obvious that distance was a much better metric for a neural network to work on, Kohli says. “It is just a fundamental part of deep learning that if you have some uncertainty associated with a decision, it is much better to have the neural network incorporate that uncertainty and decide what to do about it,” he says. Distance, unlike contact, was a richer piece of information the network could adjust and play with.

    The other twist DeepMind came up with was a second neural network that predicted the angles between amino acid pairs. With these two factors—distance and angles—DeepMind’s algorithm was able to work out a rough outline of a protein’s likely structure. It then used a different, non-A.I. algorithm to refine this structure. Putting these components together into a system it called AlphaFold, DeepMind crushed the competition in the 2018 CASP (called CASP13 because it was the 13th of the biennial contests). On the hardest set of 43 proteins in the competition, AlphaFold got the highest score on 25 of them. The next closest team scored highest on just three. The results shook the entire field: If there had been any doubt about whether deep learning methods were the most promising way to crack the protein-folding problem, AlphaFold ended them.

    Going back to the whiteboard

    Still, DeepMind was nowhere close to Hassabis’s goal: solving the protein-folding problem. AlphaFold was fairly inaccurate almost half the time. And, of the 104 protein targets in CASP13, it achieved results that were as good as X-ray crystallography in only about three cases. “We didn’t just want to be the best at this according to CASP, we wanted to be good at this. We actually want a system that matters to biologists,” Jumper says.

    No sooner had the CASP 2018 results been announced than DeepMind redoubled its efforts: Jumper was put in charge of the expanded team. Rather than simply trying to build on AlphaFold, making incremental improvements, the team went back to the whiteboard and started to brainstorm radically different ideas that they hoped would be able to bring the software closer to the kind of accuracy X-ray crystallography yielded.

    What followed, Jumper says, was one of scariest and most depressing periods of the entire project: nothing worked. “We spent three months not getting any better than our CASP13 results and starting to really panic,” he says. But then, a few of the things the researchers were trying produced a slight improvement—and within six months the system was notably better than the original AlphaFold. This pattern would continue throughout the next two years, Jumper says: three months of nothing, followed by three months of rapid progress, followed by yet another plateau.

    Hassabis says a similar pattern had occurred with previous DeepMind projects, including its work on Go and the complex, real-time strategy video game Starcraft 2. The company’s management strategy for overcoming this, he says, is to alternate between two different ways of working. The first, which Hassabis calls “strike mode,” involves pushing the team as hard as possible to wring every ounce of performance out of an existing system. Then, when the gains from the all-out effort seem to be exhausted, he shifts gears into what he calls “creative mode.” During this period, Hassabis no longer presses the team on performance—in fact, he tolerates and even expects some temporary declines—in order to give the researchers and engineers the space to tinker with new ideas and try novel approaches. “You want to encourage as many crazy ideas as possible, brainstorming,” he says. This often leads to another leap forward in performance, allowing the team to switch back into strike mode.

    A big birthday present

    On Nov. 21 of 2019, Kathryn Tunyasuvunakool, a researcher at DeepMind who works on the protein folding team, turned 30. The day would prove to be memorable for another reason too. Tunyasuvunakool, who has a Ph.D. in computational biology from the University of Oxford, was the person on the team in charge of developing new test sets for the protein-folding A.I., now dubbed AlphaFold 2, that DeepMind was developing for the 2020 CASP competition. That morning, when she turned on her office computer, she received an assessment of the system’s predictions on a batch of about 50 protein sequences—all of them only recently added to the Protein Data Bank. She did a double take. AlphaFold 2 had been improving, but on this set of proteins the results were startlingly good—predicting the structure in many cases to within 1.5 angstroms, a distance equivalent to a tenth of a nanometer, or about the width of an atom.

    Tunyasuvunakool, who calls herself “the team’s pessimist,” says her first response was not elation, but nausea. “I was feeling quite scared,” she says. The results were so good she was certain she had made a mistake—that when she was preparing the test set, she must have inadvertently allowed several proteins that the A.I. had already seen in the training data to slip in. That would have allowed AlphaFold 2 to essentially cheat, easily predicting the exact structure. Tunyasuvunakool recalls sitting in DeepMind’s cafeteria overlooking London’s St. Pancras Station and drinking cup after cup of herbal tea in an effort to calm herself. She and other team members then spent the rest of that day and late into the evening, and several days more, sitting at their workstations, painstakingly combing through AlphaFold 2’s training data to try to find the mistake.

    There wasn’t one. In fact, the new system had made a giant leap forward in performance. AlphaFold 2 was completely different from its predecessor. Rather than an assemblage of components—one to predict the distance between amino acids and another to forecast the angles, with a third piece of software to tie them together —the A.I. now used a single neural network to reason directly from the DNA sequence. While the system still took in evolutionary information—figuring out if the protein in question had a likely common ancestor to others it had seen before, and scrutinizing the alignment between the target protein’s DNA sequence and other known sequences—it no longer needed explicit data about which amino acid pairs evolved together. “Instead of providing more information, we actually provided less,” Jumper says. The system was free to draw its own insights about when ancestry might determine a portion of the protein’s shape and when it might depart more radically from that heritage. In other words, it developed a kind of intuition based on its experience, in much the same way a veteran human scientist might.

    At the heart of the new system was a mechanism called "attention." Attention, as the name implies, is a way to get a deep learning system to focus on a certain set of inputs and weigh those more heavily. For a cat identification system, for instance, the system might learn to pay attention to the shape of the ears and also learn to look for evidence of whiskers near the nose. Jumper compares what AlphaFold 2 does to the process of solving a jigsaw puzzle where “you can snap together certain pieces and be pretty sure of it, and then what you end up with are different local islands of solution, and then you figure out how to join these up.” The middle of the network, Jumper says, has learned to reason about geometry and space and how to join up those amino acid pairs it thinks are close together based on its analysis of the DNA sequences.

    DeepMind trained AlphaFold 2 on 128 “tensor processing cores,” the number-crunching brains found on 16 special computer chips engineered for deep learning that Google designed and uses in its data centers, running continuously for what the company says was a few weeks. (These 128 specialized A.I. cores are about equivalent to 100 to 200 of the powerful graphics processing chips that deliver eye-popping animation on an Xbox or PlayStation.) Once trained, the system can take a DNA sequence and spit out a complete structure prediction “in a matter of days,” the company says.

    Among AlphaFold 2’s advantages over its predecessor is a confidence gauge: The system produces a score for how sure it is of its own predictions for each amino acid in a structure. This metric is crucial if AlphaFold 2 is going to be useful to biologists and medical researchers who will need to know when they can reasonably rely on the model and when to have more caution.

    Despite the stunning test results, DeepMind was still not certain how good AlphaFold 2 was. But they got an important clue when the coronavirus pandemic struck. In March of this year, AlphaFold 2 was able to predict the structure for six understudied proteins associated with SARS-CoV-2, the virus that causes COVID-19, one of which scientists have since confirmed using an empirical method called cryogenic electron microscopy. It was a powerful glimpse of the kind of real-world impact DeepMind hopes AlphaFold 2 will soon have.

    An astonishing result

    The CASP competition takes place between May and August. The Protein Structure Prediction Center releases batches of target proteins, and contestants then submit their structure predictions for evaluation. The rankings for this year’s competition were announced on Nov. 30.

    Each prediction is scored using a metric called “global distance test total score,” or GDT for short, that in effect looks at how close, in angstroms, it is to a structure obtained by empirical methods such as X-ray crystallography or electron microscope. A score of 100 is perfect, but anything at 90 or above is considered equivalent to the empirical methods, Moult, the CASP director, says. The proteins are also classed into groups based on how difficult the CASP organizers think it is to get the structure.

    When Moult saw AlphaFold 2’s results he was incredulous. Like Tunyasuvunakool months earlier, his initial thought was that there might be a mistake. Maybe some of the protein sequences in the competition had been published before? Or maybe DeepMind had somehow managed to get hold of a cache of unpublished data?

    As a test, he asked Andrei Lupas, director of the department of protein evolution at the Max Planck Institute for Developmental Biology in Tuebingen, Germany, to conduct an experiment. Lupas would ask AlphaFold 2 to predict a structure that he knew for certain had never been seen before because Lupas had never been able to work out from X-ray crystallography what a key piece of the protein looked like. For almost a decade, Lupas had puzzled over this missing link, but the correct shape had eluded him. Now, with AlphaFold’s prediction as a guide, Lupas says, he went back to the X-ray data. “The correct structure just fell out within half an hour,” he says. “It was astonishing.”

    Since DeepMind’s success in 2018’s CASP, many academic researchers have flocked to deep learning techniques. As a result, the rest of the field’s performance has improved: On a median difficulty target, the other competitors now have an average best prediction GDT of 75, up 10 points from two years ago. But there was no comparison to AlphaFold 2: It scored a median 92 GDT across all proteins, and even on the most difficult proteins it achieved a median score of 87 GDT. Moult says AlphaFold 2’s predictions are “on par with empirical methods,” such as X-ray crystallography. That conclusion lead CASP to make a momentous declaration on Monday, Nov. 30: The 50-year-old protein-folding problem had been solved.

    Venki Ramakrishnan, a Nobel Prize–winning structural biologist who is also the current president of The Royal Society, Britain’s most prestigious scientific body, says AlphaFold 2 “represents a stunning advance” in protein folding. With AlphaFold 2, expensive and time-consuming empirical analysis with methods like X-ray crystallography and electron microscopes may become a thing of the past.

    Janet Thornton, an expert in protein structure and former director of the European Molecular Biology Laboratory’s European Bioinformatics Institute, says that DeepMind’s breakthrough will allow scientists to map the entire human “proteome”—all the proteins found within the human body. Currently only a quarter of these proteins have been used as targets for drugs, but having the structure for the rest would create vast opportunities for the development of new therapies. She also says the A.I. software could enable protein engineering that might aid in sustainability efforts, allowing scientists to potentially create new crop strains that provide more nutritional value per acre of land planted, and also possibly allowing for the advent of enzymes that could digest plastic.

    For now, though, the question remains about how exactly DeepMind will make AlphaFold 2 available. Hassabis says the company is committed to ensuring the software can “make the maximal positive societal impact.” But he says it has not yet determined how to do that, saying only that it will make an announcement sometime next year. Hassabis also tells Fortune that DeepMind is considering how it might be able to build a commercial product or partnership around the system. “This should be hugely useful for the drug discovery process and therefore Big Pharma,” he says. But exactly what form this commercial offering will take, he says, has not yet been decided either.

    A commercial venture would be marked departure for DeepMind, which, since its sale to Alphabet, has not had to worry about generating revenue. The company briefly set up a division called DeepMind Health that was working with the U.K.’s National Health Service on an app that could identify hospital patients who were at risk of developing acute kidney injury. But the effort became embroiled in a controversy after news reports revealed DeepMind's hospital partner had violated the U.K. data protection laws by giving the company access to millions of patients’ medical records. In 2019, DeepMind Health was formally absorbed into a new Google health division. At the time, DeepMind said cleaving off its health effort would allow it to remain true to its research roots without the distraction of having to build a commercial unit that might replicate areas, such as data security and customer support, where Google already had expertise.

    Of course, if DeepMind were to launch a commercial product, it would not be the first A.I. research company to do so: OpenAI, the San Francisco–based research company that is perhaps DeepMind’s closest rival, has become increasingly business-oriented. Last year, OpenAI launched its first commercial product, an interface that lets companies use an A.I. that composes long passages of coherent text from a short, human-written prompt. The business value of that A.I., called GPT-3, remains unproven, while DeepMind’s AlphaFold 2 could have an immediate bottom-line impact for a pharmaceutical company or biotechnology startup. At a time when antitrust regulators are probing Alphabet, having a viable commercial product could be a good insurance policy for DeepMind in the event it ever loses the unconditional support of its deep-pocketed parent in some future breakup of the Googleplex.

    One thing is certain: DeepMind isn’t done with protein folding. The CASP competition was set up around predicting the structure of single proteins. But in biology and medicine, it is usually protein interactions that researchers really care about. How does one protein bind with another or with a particular small molecule? Exactly how does an enzyme break a protein apart? The problem of predicting these interactions and bindings will likely become the primary focus of future CASP competitions, Moult says. And Jumper says DeepMind plans to work on those challenges next.

    Reverberations from AlphaFold 2’s success are certain to be felt in areas far removed from protein folding, too, encouraging others to apply deep learning to big scientific questions: finding new subatomic particles, probing the secrets of dark matter, mastering nuclear fusion, or creating room-temperature superconductors. DeepMind has an active effort already underway on astrophysics, Kohli says. Facebook’s A.I. researchers just launched a deep learning project aimed at finding new chemical catalysts. Protein folding is the first foundational scientific mystery to fall to the power of artificial intelligence. It won’t be the last.

    最新:
    • 熱讀文章
    • 熱門視頻
    活動
    掃碼打開財富Plus App
    快3玩法绝招