Hướng dẫn Robots.txt xeforo chuẩn seo 2016

[tionsec]

Robots.txt xeforo chuẩn seo 2016 cho ae, Mong AE Đừng Chém
Tham Khảo tại Đây Nhé : https://anonsec.gq/robots.txt
Vài Ngày Nữa Cập Nhật Thêm yandex

 

[THB]

nhiều nhỉ? để hôm nào thử.
thanks

User-agent: *
Disallow: /

User-agent: Googlebot
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-Agent: Googlebot-Mobile
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-Agent: Googlebot-Image
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-Agent: Googlebot-News
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-Agent: facebookexternalhit
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-agent: msnbot
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-agent: msnbot-media
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-agent: bingbot
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-agent: Slurp
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-agent: ia_archiver
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-agent: pagespeedbot
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-agent: Websnapr
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

User-agent: Twitterbot
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /beobachtete/
Disallow: /chat/
Disallow: /conversations/
Disallow: /find-new/
Disallow: /finde-neue/
Disallow: /help/
Disallow: /hilfe/
Disallow: /letzte-aktivitaeten/
Disallow: /login/
Disallow: /lost-password/
Disallow: /members/
Disallow: /misc/
Disallow: /mitglieder/
Disallow: /mobiquo/
Disallow: /online/
Disallow: /profil/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /unterhaltungen/
Disallow: /watched/
Disallow: /info/

 

[tionsec]

Bác Cứ Tự Nhiên

 

[KenLilBoiz]

chặn thế này thì ai dám vào website bạn trao đổi thường xuyên nữa ??? không cho người ta đặt link ở trong profile thì ai dám đăng ký ) làm forum giờ chủ yếu người ta đi link ở profile là nhiều -> cái níu kéo tốt nhất để người khác chăm vào thảo luận hoặc tìm đến website thì lại chặn :v
p/s theo quan điểm của riêng mình thì không nên chặn. hì

 

[SangNTP]

Sẵn tiện nhờ ace giải thích hộ nếu forum có robots.txt và không có thì có gì khác nhau?
Mình đang học hỏi.

 

[lamtraumvn]

sao bác disallow nhiều thế, info còn chặn nữa thì bot làm sao craw được dữ liệu ở profile. Không có link ở profile là một thiệt thòi lớn cho nhiều mem đấy

 

[tionsec]

chặn thế này thì ai dám vào website bạn trao đổi thường xuyên nữa ??? không cho người ta đặt link ở trong profile thì ai dám đăng ký ) làm forum giờ chủ yếu người ta đi link ở profile là nhiều -> cái níu kéo tốt nhất để người khác chăm vào thảo luận hoặc tìm đến website thì lại chặn :v
p/s theo quan điểm của riêng mình thì không nên chặn. hì

Web hacking bác, Securyti Là Trên Cùng

 

[tionsec]

em sẽ gỡ

 

[Lê Tí]

Chặn thế không tốt đâu bác

 

[azorakt]

Mà chèn chỗ nào trên Xenforo vậy các anh.

 

[tionsec]

Tại forum này có phải đi link đồ gì đâu

 

[tionsec]

Mà chèn chỗ nào trên Xenforo vậy các anh.

Mở host ra, tạo file robots.txt , up ngang index

 

[Jindo_Katori]

Mình cũng chặn kiểu này, chặn hết