I'm trying to scrape a website using the package web_scraper, where I want the user to click a button and the new link opens where new scrapped images can be shown.
class Top2 extends StatefulWidget {
const Top2({Key? key}) : super(key: key);
#override
State<Top2> createState() => _Top2State();
}
class _Top2State extends State<Top2> {
late List<Map<String, dynamic>> top2Wall;
bool top2Loaded = false;
int page = 2;
void top2Fetch() async {
final top2Scraper = WebScraper('https://mobile.alphacoders.com');
if (await top2Scraper
.loadWebPage('/by-category/3?page=$page&quickload=1')) {
top2Wall = top2Scraper.getElement('div.item > a > img', ['src', 'title']);
// ignore: avoid_print
print(top2Wall);
setState(() {
top2Loaded = true;
});
}
}
#override
void initState() {
super.initState();
top2Fetch();
}
#override
Widget build(BuildContext context) {
Size screenSize = MediaQuery.of(context).size;
return Scaffold(
body: top2Loaded
// ignore: sized_box_for_whitespace
? Container(
height: screenSize.height,
width: double.infinity,
child: SingleChildScrollView(
physics: const BouncingScrollPhysics(),
child: Wrap(children: [
for (int i = 1; i < top2Wall.length; i++)
WallCard(src: top2Wall[i]['attributes']['src']),
InkWell(
onTap: () {
setState(() {
page++;
// ignore: avoid_print
print(page);
});
},
child: Container(
height: 100,
width: 100,
decoration: const BoxDecoration(color: Colors.cyan)),
)
]),
),
)
: const Center(
child: CircularProgressIndicator(color: Colors.cyanAccent),
));
}[image2][1]
}
For a clear explanation I want to change the page link for scraping, so basically the page number=1 i want to increase it by a number, when user clicks on the container. I used SetState and page++ to increment it by a digit everytime the user clicks on it. I used the print statement to check wether the page increments or not and its sucessfully increases but the page remains same, for clear code view please refer these images enter image description here
enter image description here
If I understand what your issue is, after calling increment page ++ in setstate, you have to call top2Fetch() again inside the onTap method. Like this:
onTap: ()async {
setState(()=>page++);
await top2Fetch();
}
Then update your UI.
You can include a bool value called isLoading in the onTap method: such that when you call the top2Fetch method , a loading spinner is shown on the UI till the future is done.
Related
I want to scrap a lazy loaded website using flutter, i used webscraper package as it only scrapes visible elements, how can I scrap all the images and links from the website.
Please refer the image
class Top2 extends StatefulWidget {
const Top2({Key? key}) : super(key: key);
#override
State<Top2> createState() => _Top2State();
}
class _Top2State extends State<Top2> {
late List<Map<String, dynamic>> top2Wall;
bool top2Loaded = false;
void top2Fetch() async {
final top2Scraper = WebScraper('https://mobile.alphacoders.com');
if (await top2Scraper.loadWebPage('/by-category/3?page=1')) {
top2Wall = top2Scraper.getElement(
'div.container-masonry > div.item > a > img.img-responsive',
['src', 'title']);
// ignore: avoid_print
print(top2Wall);
setState(() {
top2Loaded = true;
});
}
}
#override
void initState() {
super.initState();
top2Fetch();
}
#override
Widget build(BuildContext context) {
Size screenSize = MediaQuery.of(context).size;
return Scaffold(
body: top2Loaded
// ignore: sized_box_for_whitespace
? Container(
height: screenSize.height,
width: double.infinity,
child: SingleChildScrollView(
physics: const BouncingScrollPhysics(),
child: Wrap(children: [
for (int i = 1; i < top2Wall.length; i++)
WallCard(src: top2Wall[i]['attributes']['src'])
]),
),
)
: const Center(
child: CircularProgressIndicator(color: Colors.cyanAccent),
));
}
}
So basically i want to scrap all the wallpaper available from the website but can only scrap the first 24 wallpapers visible.
Its simple.
Rest of the wallpapers are fetched dynamic using JS.
Check Pc Browser > Dev Options> Network and scroll down the page you will se something like:
You can maybe check url where XHR connect and try to scrap it
EDIT
I See page have pagination switch so maybe will be easier to use this
I wanted to ask this certain question, I don't know if I'm being silly or not.
[https://mobile.alphacoders.com/by-category/10?page=1&quickload=1]
here you can see in the link "page=1", I want to create a button in my flutter app so that when I click on the button the page number increases with one digit
so like page=1+1
I think I can use the '$' property in the URL and add a setState but the trickier thing is I'm trying to scrape a website's wallpaper so when the user clicks on the next page button he must be redirected to a new page with new wallpapers
class Top2 extends StatefulWidget {
const Top2({Key? key}) : super(key: key);
#override
State<Top2> createState() => _Top2State();
}
class _Top2State extends State<Top2> {
late List<Map<String, dynamic>> top2Wall;
bool top2Loaded = false;
void top2Fetch() async {
final top2Scraper = WebScraper('https://mobile.alphacoders.com');
if (await top2Scraper.loadWebPage('/by-category/3?page=1')) {
top2Wall = top2Scraper.getElement(
'div.container-masonry > div.item > a > img.img-responsive',
['src', 'title']);
// ignore: avoid_print
print(top2Wall);
setState(() {
top2Loaded = true;
});
}
}
#override
void initState() {
super.initState();
top2Fetch();
}
#override
Widget build(BuildContext context) {
Size screenSize = MediaQuery.of(context).size;
return Scaffold(
body: top2Loaded
// ignore: sized_box_for_whitespace
? Container(
height: screenSize.height,
width: double.infinity,
child: SingleChildScrollView(
physics: const BouncingScrollPhysics(),
child: Wrap(children: [
for (int i = 1; i < top2Wall.length; i++)
WallCard(src: top2Wall[i]['attributes']['src'])
]),
),
)
: const Center(
child: CircularProgressIndicator(color: Colors.cyanAccent),
));
}
}
please refer to this image for clear code sample
Hello #Cheems Monarch
You can try like this :
First create a variable:
int page=1;
Then change your method like this:
void top2Fetch(String page) async {
final top2Scraper = WebScraper('https://mobile.alphacoders.com');
if (await top2Scraper.loadWebPage('/by-category/3?page=$page')) {
top2Wall = top2Scraper.getElement(
'div.container-masonry > div.item > a > img.img-responsive',
['src', 'title']);
// ignore: avoid_print
print(top2Wall);
setState(() {
top2Loaded = true;
});
}
}
And call this function on your button's onPressed:
onPressed: () {
top2Fetch(page.toString());
setState(() {
page++;
});
}
I am using provider for state management, and I've given onTap a value in a function in the ChangeNotifier child class but my app is unresponsive, I mean, when i tap on the widget, it doesn't update state, however, it does change the values i need it to change tho, i know this coz I am debugPrinting in the onTap function and when i tap, it actually prints that the button got tapped, but state doesn't update, widget remains the same until i hot restart, then it updates everything, even hot reload doesn't update it, here's the function
class Storage extends ChangeNotifier{
static const _storage = FlutterSecureStorage();
static const _listKey = 'progress';
List _dataMaps = [];
List<DayTile> dayTileMain = [];
void createDataMap() {
for (int i = 1; i < 101; i++) {
final data = Data(number: i).toJson();
_dataMaps.add(data);
}
}
void createDayTiles() {
for(Map<String, dynamic> data in _dataMaps) {
bool isDone = data['i'];
final dayTile = DayTile(
number: data['n'],
isDone: isDone,
// This is where i need to rebuild the tree
onTap: () async {
data['i'] = true;
notifyListeners();
print(data['i']);
print(isDone);
await writeToStorage();
},
);
dayTileMain.add(dayTile);
}
print('data tiles created');
}
}
and here is the DayTile class
class DayTile extends StatelessWidget {
const DayTile({
Key? key,
required this.number,
required this.isDone,
required this.onTap,
}) : super(key: key);
final int number;
final VoidCallback onTap;
final bool isDone;
#override
Widget build(BuildContext context) {
return GestureDetector(
onTap: onTap,
child: Container(
height: 50,
width: MediaQuery.of(context).size.width * .15,
decoration: BoxDecoration(
color: !isDone
? const Color(0xffedecea)
: const Color(0xffedecea).withOpacity(0.1),
borderRadius: BorderRadius.circular(5),
),
child: Center(
child: Stack(
alignment: Alignment.center,
children: [
Center(
child: Text(
number.toString(),
style: const TextStyle(
color: Color(0xff576aa4),
),
),
),
Visibility(
visible: isDone,
child: const Divider(
color: Colors.black,
),
),
],
),
),
),
);
}
}
here is where I listen for the change
Wrap(
spacing: 13,
runSpacing: 13,
children: Provider.of<Storage>(context).dayTileMain,
),
when data['i'] is true, it should update the current instance of DayTile() that it's on in the loop, and in DayTile() I use the value of data['i'] to set the value of bool isDone and depending on whether isDone is true or false, the color of the widget changes and some other things, BUT, they don't change onTap, but they change after I hot restart, when it's read the storage and restored the saved data, could the secureStorage writing to storage at the same time be affecting it?
I solved it, turns out it's not a good idea to listen for events in the model class, it won't listen, so instead of generating the list of widgets in the model class, I moved it outta there, and instead generated it inside the wrap widget, and instead of listening for a list in the model class, i just had the list there in my wrap, if it was a listview i was tryna generated, i would've done this initially with a ListView.builder() i didn't know you could generate a list inside the children of the wrap widget, so i just stuck to defining it in the model, I came across this stack question Flutter: How to use Wrap instead of ListView.builder?
and that's how i knew how to build widgets inside a children property, i was actually just looking for a ListView.builder() version for the Wrap widget, all said, this is what my stuff is looking like
Model
class Storage extends ChangeNotifier {
static const _storage = FlutterSecureStorage();
static const _listKey = 'progress';
List _dataMaps = [];
List<DayTile> dayTileMain = [];
void createDataMap() {
for (int i = 1; i < 101; i++) {
final data = Data(number: i).toJson();
_dataMaps.add(data);
}
}
int get listLength {
return _dataMaps.length;
}
UnmodifiableListView get dataMaps {
return UnmodifiableListView(_dataMaps);
}
void pressed(Map<String, dynamic> map) async {
map['i'] = true;
await writeToStorage();
notifyListeners();
}
Future writeToStorage() async {
final value = json.encode(_dataMaps);
await _storage.write(key: _listKey, value: value);
}
Future<void> getTasks() async {
print('getTasks called');
final value = await _storage.read(key: _listKey);
final taskList = value == null ? null : List.from(jsonDecode(value));
if (taskList == null) {
print('getTasks is null');
createDataMap();
// createDayTiles();
} else {
print('getTasks is not null');
print(taskList);
_dataMaps = taskList;
// createDayTiles();
}
}
Future readFromStorage() async {
await getTasks();
notifyListeners();
}
}
Wrap Builder
class DayTiles extends StatelessWidget {
const DayTiles({Key? key}) : super(key: key);
#override
Widget build(BuildContext context) {
return Consumer<Storage>(
builder: (_, storageData, __) => Wrap(
spacing: 13,
runSpacing: 13,
children: [
for(Map<String, dynamic> data in storageData.dataMaps)
DayTile(
number: data['n'],
onTap: () {
storageData.pressed(data);
},
isDone: data['i'],
),
],
),
);
}
}
and instead of using a wrap and listening for changes to it's children in the screen class, i just directly use the DayTiles() custom widget i created
I am a beginner to rive and flutter. I am building a favorite items page in flutter. If there are not anything added to favorites I need to show a riveAnimation on screen. I already implemented almost everything to show the animation on screen. But I need to toggle a jumping animation when user tap on the animation which is really cool. for now I have the animation on 'Idle' mode
You may want to refer to the rive file => Go to Rive. And I renamed Rive stateMachine name to Bird. Everything else is the same.
summary => I want bird to jump when user tap on him :)
The code and the image may be little bit bigger. Sorry about that
class Favourites extends StatefulWidget {
Favourites({Key? key}) : super(key: key);
#override
State<Favourites> createState() => _FavouritesState();
}
class _FavouritesState extends State<Favourites> {
String animation = 'idle';
SMIInput<String>? _birdInput;
Artboard? _birdArtboard;
void jump() {
setState(() {
_birdInput?.value = 'Pressed';
});
}
#override
void initState() {
super.initState();
rootBundle.load('assets/rive/bird.riv').then(
(data) {
final file = RiveFile.import(data);
final artboard = file.mainArtboard;
var controller = StateMachineController.fromArtboard(
artboard,
'Bird',
);
if (controller != null) {
artboard.addController(controller);
_birdInput = controller.findInput('Pressed');
}
setState(() => _birdArtboard = artboard);
},
);
}
#override
Widget build(BuildContext context) {
final favourite = Provider.of<Favourite>(context);
return Scaffold(
backgroundColor: Colors.grey[300],
appBar: const CustomAppBar(title: 'Favourites'),
body: favourite.items.isEmpty
? Center(
child: Column(
children: [
SizedBox(
width: 300,
height: 500,
child: _birdArtboard == null
? const SizedBox()
: Center(
child: GestureDetector(
onTap: () {},
child: Rive(artboard: _birdArtboard!),
),
),
),
NeumorphicButton(),
],
),
)
: CustomGrid(),
);
}
}
If you open/run rive file on rive site, you can find that it is using Trigger variable for jumping and it is using State Machine 1 state machine.
Next thing comes about declaring variable. You need to use SMITrigger data type for this and use StateMachineController to control the animation.
Use .findSMI(..) instead of .findInput() for SMITrigger.
To start animation on trigger, use
trigger?.fire();
I will encourage you to take a look on editor and check input variable type while performing rive animation.
So the full widget that will provide animation is
class Favourites extends StatefulWidget {
const Favourites({Key? key}) : super(key: key);
#override
State<Favourites> createState() => _FavouritesState();
}
class _FavouritesState extends State<Favourites> {
String animation = 'idle';
Artboard? _birdArtboard;
SMITrigger? trigger;
StateMachineController? stateMachineController;
#override
void initState() {
super.initState();
rootBundle.load('assets/rive/bird.riv').then(
(data) {
final file = RiveFile.import(data);
final artboard = file.mainArtboard;
stateMachineController =
StateMachineController.fromArtboard(artboard, "State Machine 1");
if (stateMachineController != null) {
artboard.addController(stateMachineController!);
trigger = stateMachineController!.findSMI('Pressed');
stateMachineController!.inputs.forEach((e) {
debugPrint(e.runtimeType.toString());
debugPrint("name${e.name}End");
});
trigger = stateMachineController!.inputs.first as SMITrigger;
}
setState(() => _birdArtboard = artboard);
},
);
}
void jump() {
trigger?.fire();
}
#override
Widget build(BuildContext context) {
return Scaffold(
backgroundColor: Colors.grey[300],
body: Center(
child: Column(
children: [
SizedBox(
width: 300,
height: 400,
child: _birdArtboard == null
? const SizedBox()
: Center(
child: GestureDetector(
onTap: () {
jump();
},
child: Rive(artboard: _birdArtboard!),
),
),
),
],
),
));
}
}
On the home page I have 4 different api calls fetching 4 different data from a wordpress site. The current way I have coded is that everytime we enter home page, in the initState(), the get _getHomePageData() function is called where the async api fetching is happening. while this is happening allDataLoaded boolean is set to false in the first place.
Once the data is loaded allDataLoaded is set to true, the loading stops and the widgets are shown.
Here is the homepage widget:
class HomePage extends StatefulWidget {
#override
_HomePageState createState() => _HomePageState();
}
class _HomePageState extends State<HomePage> {
final String homePageLatestArtworkApi =
'https://example.com/wp-json/wp/v2/artworks?
per_page=1&_embed';
final String homePageAllArtworkApi =
'https://example.com/wp-json/wp/v2/artworks?
per_page=6&_embed';
final String homePageAllEventsApi =
'https://example.com/wp-json/wp/v2/events?
per_page=6&_embed';
final String homePageAllVenuesApi =
'https:example.com/wp-json/wp/v2/venues?
per_page=6&_embed';
List homePageLatestArtwork;
List homePageAllArtworks;
List homePageAllEvents;
List homePageAllVenues;
var allDataLoaded = false;
#override
void initState(){
super.initState();
print('init state is called everytime the page is loaded');
_getHomePageData();
}
Future<String> _getHomePageData() async{
var responseLatestArtwork = await http.get(Uri.encodeFull(homePageLatestArtworkApi));
var responseAllArtworks = await http.get(Uri.encodeFull(homePageAllArtworkApi));
var responseAllEvents = await http.get(Uri.encodeFull(homePageAllEventsApi));
var responseAllVenues = await http.get(Uri.encodeFull(homePageAllVenuesApi));
setState(() {
//latest artwork
var convertDataToJsonLatestArtwork = json.decode(responseLatestArtwork.body);
homePageLatestArtwork = convertDataToJsonLatestArtwork;
//All Artworks
var convertDataToJsonAllArtworks = json.decode(responseAllArtworks.body);
homePageAllArtworks = convertDataToJsonAllArtworks;
// All Events
var convertDataToJsonAllEvents = json.decode(responseAllEvents.body);
homePageAllEvents = convertDataToJsonAllEvents;
//All venues
var convertDataToJson = json.decode(responseAllVenues.body);
homePageAllVenues = convertDataToJson;
if(homePageLatestArtwork != null && homePageAllArtworks != null && homePageAllEvents != null && homePageAllVenues != null){
allDataLoaded = true;
}
// print('converted data is here');
//print(homePageLatestArtwork);
//print('the title is here :');
//print(homePageLatestArtwork[0]['title']['rendered']);
//print(homePageLatestArtwork[0]['_embedded']['wp:featuredmedia'][0]['source_url']);
});
#override
Widget build(BuildContext context) {
if(allDataLoaded){ //wait for the data to load and show spinning loader untill data is completely loaded
return Scaffold(
// body: Text('title'),
// body: Text("title: ${homePageLatestArtwork[0]['title']['rendered']} "),
body: Container(
child: SingleChildScrollView(
child: Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: <Widget>[
// Expanded(
Container(
height: 200.0,
child: Image.network(homePageLatestArtwork[0]['_embedded']['wp:featuredmedia'][0]['source_url'],
fit: BoxFit.fill,
width: double.maxFinite,),
),
Container(
padding: EdgeInsets.only(left:15.0,right:15.0,top:10.0),
child: Text(homePageLatestArtwork[0]['title']['rendered'],textAlign: TextAlign.left,
style: new TextStyle(
fontSize: 20.0,
fontFamily: 'Montserrat-Regular',
color: Color(0XFF212C3A),
fontWeight: FontWeight.bold
),),
),
])
),
),
);
}else{
return new Center(
child: new CircularProgressIndicator(
valueColor: new AlwaysStoppedAnimation<Color> .
(Theme.of(context).primaryColor),
),
);
}
} //end of _HOMEPAGESTATE
I would like to not load every single time the home page is viewed, or let's say I just want to fetch the api data once when the app starts and let users manually pull down to refresh the data.
Method 1:
If you are just talking about data then you should have a singleton object that you can init once in initState();
class _HomePageData {
var allDataLoaded = false;
String contents = "";
Future<String> _getHomePageData() async {
// Assuming contents is the data model and you load the data into contents
this.contents = "Loaded";
this.allDataLoaded = true;
return this.contents;
}
}
final _HomePageData data = _HomePageData();
class _HomePageState extends State<HomePage> {
String contents;
#override
void initState(){
super.initState();
if (!data.allDataLoaded) {
data._getHomePageData().then((contents) {
setState(() {
this.contents = contents;
})
});
} else {
this.contents = data.contents;
}
}
}
Method 2:
Previously I was dealing with tabs that constantly reload every time I move to another tab. Flutter actively remove state when they are not attached to any active widget tree. The idea is to have the widget in a Stack, placed into Offstage & TickerMode to control visibility and animation.
class MyTabState extends State<MyTabPage> {
#override
Widget build(BuildContext context) {
return Scaffold(
bottomNavigationBar: MyBottomBar(
onTab: _onTab,
currentIndex: _currentIndex,
),
body: Stack(
children: List.generate(3, (index) {
return Offstage(
offstage: _currentIndex != index,
child: TickerMode(
enabled: _currentIndex == index,
child: getChild(index),
),
);
}, growable: false),
),
);
}
}
I hope I am not enough late. Anyhow if anyone else get the same situation not to refresh again and again you can add the following line at the end of your state. For example
with AutomaticKeepAliveClientMixin
It should be like this:
class _HomeScreenState extends State<HomeScreen>
with AutomaticKeepAliveClientMixin
then you can add an override
#override
bool get wantKeepAlive => true;
You are ready to rock.